ReWOO proposes a novel paradigm for Augmented Language Models (ALMs) that decouples the reasoning process from external tool observations, aiming to significantly reduce token consumption and computational complexity. This approach is beneficial for researchers and developers building efficient, tool-augmented AI systems, particularly those seeking to optimize LLM interactions and enable smaller models to perform complex reasoning tasks.

How It Works

ReWOO separates the LLM's reasoning steps from the actual execution and observation of tool outputs. Unlike interleaved methods like ReAct, ReWOO first generates a complete reasoning plan, then executes the necessary tools, and finally uses the observations to refine the plan. This decoupling allows for more efficient prompt usage and can potentially offload reasoning capabilities from larger LLMs to smaller, fine-tuned models.

Quick Start & Requirements

• Install: pip install -r requirements.txt
• Prerequisites: OpenAI, WolframAlpha, and SerpApi API keys stored in ./keys/.
• Run a single instance: python run.py --method rewoo --toolset Google LLM --base_lm text-davinci-003
• Evaluate on benchmarks: python run_eval.py --method rewoo --dataset hotpot_qa --sample_size 10 --toolset Wikipedia LLM --base_lm gpt-3.5-turbo --save_result
• Demo: Live Demo (ReWOO integrated into Gentopia)

Highlighted Details

• Achieves 5x token efficiency and 4% accuracy improvement on HotpotQA.
• Demonstrates robustness under tool-failure scenarios.
• Enables offloading reasoning from 175B GPT-3.5 to a 7B LLaMA model.
• Supports various benchmarks including HotpotQA, TriviaQA, GSM8K, and more.

Maintenance & Community

The project's official implementation has been integrated into Gentopia. Further details and community interaction may be available through Gentopia's channels.

Licensing & Compatibility

The README does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The repository is described as the "original implementation" with a "better" version integrated into Gentopia, suggesting potential differences or deprecation of this specific repo. API key management is required for functionality.