AnyTool implements a self-reflective, hierarchical agent system designed for complex, large-scale API calls. It targets researchers and developers working with LLM-based agents that need to interact with diverse APIs, offering a structured approach to improve task completion and accuracy.

How It Works

AnyTool employs a hierarchical agent architecture with a self-reflective mechanism. It breaks down complex queries into smaller, manageable sub-tasks, leveraging a "Solver" component (e.g., GPT-4 with DFSDT) to generate API calls and interpret results. The self-reflection capability allows the agent to iteratively refine its plan and API usage based on intermediate outcomes, enhancing robustness and accuracy in multi-API scenarios.

Quick Start & Requirements

• Install: pip install -r requirements.txt
• Prerequisites: Python 3.9+, OpenAI API (GPT-4 recommended, Azure OpenAI used in experiments), ToolBench key, ToolBench data.
• Setup: Requires downloading and decompressing ToolBench data, configuring API keys in config.py, and running preprocessing scripts.
• Links: ToolBench

Highlighted Details

• Achieves 58.2% pass rate on filtered ToolBench (G1-G3) and 73.8% on AnyToolBench with SR GPT-4.
• Supports multiple API retrieval methods and solver configurations.
• Includes scripts for generating AnyToolBench data and running experiments.
• Provides detailed benchmark results comparing AnyTool against other LLM agent frameworks.

Maintenance & Community

This project is built upon the ToolBench framework. No specific community channels or active maintainer information beyond the cited authors are provided in the README.

Licensing & Compatibility

The repository does not explicitly state a license. It relies on the ToolBench dataset and OpenAI APIs, which have their own terms of service. Commercial use may be restricted by these underlying dependencies.

Limitations & Caveats

The system heavily relies on specific OpenAI models (GPT-4) and the ToolBench dataset, potentially limiting generalizability. Setup involves significant data preparation and API key configuration. The "SR Agent" and "SR GPT-4" components are specific to this implementation and may require further research for independent replication.