This repository provides a preliminary implementation for improving language model factuality and reasoning via multi-agent debate, as detailed in an ICML 2024 paper. It targets researchers and practitioners in NLP and AI seeking to enhance LLM performance on complex tasks.

How It Works

The core approach involves setting up a debate between multiple language model agents to refine answers and improve factual accuracy. Agents present arguments, critique each other, and converge on a more robust solution, leveraging a multi-agent system architecture.

Quick Start & Requirements

• Install: Navigate to specific task directories (e.g., ./math/, ./gsm/) and run Python scripts.
• Prerequisites: Python, specific datasets (GSM, MMLU) may need to be downloaded.
• Links: Project Page, Paper.

Highlighted Details

• Implements multi-agent debate for improving factuality and reasoning in LLMs.
• Supports tasks including arithmetic, GSM, biographies, and MMLU.
• Includes scripts for both generating debate-based answers and evaluating results.

Maintenance & Community

This is a preliminary implementation with plans for more tasks and settings. No community links or active maintenance signals are provided in the README.

Licensing & Compatibility

The repository does not explicitly state a license.

Limitations & Caveats

This is a preliminary implementation, and further tasks and settings are planned for future release. The lack of an explicit license may pose compatibility concerns for commercial or closed-source use.