This repository presents LLM-ToolMaker (LATM), a framework enabling Large Language Models (LLMs) to autonomously create and utilize reusable Python tools for problem-solving. It targets researchers and developers seeking to enhance LLM capabilities beyond their inherent knowledge, offering a cost-effective method for complex reasoning tasks by separating tool creation from tool application.

How It Works

LATM operates in two phases: tool making and tool using. In the tool-making phase, an LLM (the "tool maker") generates Python utility functions based on task demonstrations. This involves proposing a tool, verifying its correctness with unit tests, and wrapping it for usability. The tool-using phase employs an LLM (the "tool user") to translate natural language queries into function calls, leveraging the previously created tools. This separation allows for using a powerful model for tool creation and a more efficient model for tool execution, optimizing cost and performance.

Quick Start & Requirements

• Install: pip install -r requirements.txt (from the root directory)
• Prerequisites: Python 3.8+, OpenAI API key (for GPT models).
• Resources: Requires access to LLM APIs (e.g., GPT-4, GPT-3.5).
• Documentation: Paper link

Highlighted Details

• Enables LLMs to generate reusable Python tools, reducing reliance on pre-existing APIs.
• Achieves performance comparable to using a high-end model for both tasks by employing a powerful tool maker and a lightweight tool user.
• Validated on complex reasoning tasks, including Big-Bench benchmarks.
• Tool-making pipeline includes tool proposing, verification (unit tests), and wrapping.

Maintenance & Community

The project is associated with authors from Microsoft Research. No specific community channels or roadmap are detailed in the README.

Licensing & Compatibility

The repository's license is not explicitly stated in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The framework relies on LLM API access, specifically mentioning GPT models, which may incur costs. The effectiveness and robustness of generated tools are dependent on the capabilities of the LLM used as the tool maker.