Gorilla is a suite of tools and models designed to enable Large Language Models (LLMs) to accurately invoke APIs and interact with external services. It addresses the challenge of LLMs hallucinating or incorrectly calling APIs by providing specialized models, a comprehensive API benchmark dataset (APIBench), and evaluation frameworks. The project targets developers building AI agents, applications requiring LLM-tool integration, and researchers evaluating LLM function-calling capabilities.

How It Works

Gorilla utilizes a novel fine-tuning approach that incorporates retrieval-augmented training on a massive collection of API documentation (APIBench). This method allows the LLM to learn the nuances of API signatures, parameters, and usage patterns, significantly reducing hallucinations and improving accuracy in API invocation. The project also offers an execution engine (GoEx) for safe execution of LLM-generated actions, featuring post-facto validation and damage confinement.

Quick Start & Requirements

• Gorilla CLI: pip install gorilla-cli
• Local Inference: git clone the repository and follow detailed local setup instructions.
• OpenFunctions Usage: Requires Python with openai library. API base URL and model name are specified in code.
• Dependencies: Python, openai library. Specific model requirements (e.g., GPU) are not explicitly detailed for basic inference but are implied for training/fine-tuning.
• Demos: Colab notebooks and Gradio interfaces are available for trying base and OpenFunctions models.
• Resources: Links to Hugging Face models and a live leaderboard are provided.

Highlighted Details

• Supports over 1,600 APIs with ongoing expansion via API Zoo.
• OpenFunctions V2 offers state-of-the-art performance for open-source models, supporting multiple and parallel function execution with expanded data types.
• Berkeley Function Calling Leaderboard (BFCL) provides comprehensive evaluation, including multi-turn and multi-step capabilities, cost, and latency metrics.
• Agent Arena allows comparison of LLM agents across various models and tools.

Maintenance & Community

The project is actively maintained with frequent updates to the Berkeley Function Calling Leaderboard (BFCL) and new releases like Agent Arena and GoEx. Community engagement is encouraged via a Discord server and contributions to the API Zoo.

Licensing & Compatibility

Gorilla models are released under the Apache 2.0 license, permitting commercial use. This makes it compatible with closed-source applications.

Limitations & Caveats

While the project provides extensive evaluation and tooling, specific hardware requirements for running larger models or fine-tuning are not explicitly detailed in the README. The roadmap indicates ongoing development for multi-turn and multimodal function calling, suggesting these capabilities may still be evolving.