Empower any LLM with function calling capabilities by acting as a middleware proxy. Toolify injects OpenAI-compatible function calling into models or APIs that do not natively support it, enabling developers to integrate advanced conversational AI features into their applications. It provides universal function calling, supports multiple calls, streaming, and robust error handling.

How It Works

Toolify acts as an intermediary, intercepting client requests containing tool definitions. It injects a custom system prompt instructing the LLM to format function calls using a specific XML structure and trigger signal. The modified request is proxied to an upstream LLM. Toolify then parses the LLM's response for the trigger and XML, transforming extracted function calls into the standard OpenAI tool_calls format before returning them to the client.

Quick Start & Requirements

Installation is supported via Docker Compose (recommended) or directly using Python 3.8+. Docker users clone the repo (https://github.com/funnycups/toolify.git), configure config.yaml, and run docker-compose up -d. Python users clone, install dependencies (pip install -r requirements.txt), configure config.yaml, and run python main.py. Both methods require editing config.yaml to define upstream LLM services and client authentication keys. The service is accessible at http://localhost:8000.

Highlighted Details

• Universal function calling for LLMs and OpenAI-compatible APIs lacking native support.
• Supports simultaneous multiple function calls and flexible initiation.
• Gracefully handles <think> tags and fully supports streaming responses.
• Features multi-service routing, client API key authentication, and enhanced context awareness for tool results.
• Provides accurate token counting and automatic retries on function call parsing failures.

Maintenance & Community

No specific details regarding maintainers, community channels (e.g., Discord, Slack), or roadmap are provided in the README.

Licensing & Compatibility

Licensed under GPL-3.0-or-later, a strong copyleft license. This necessitates that derivative works also be licensed under GPL-3.0-or-later, potentially restricting integration into closed-source commercial products.

Limitations & Caveats

The primary caveat is the GPL-3.0-or-later license, which imposes copyleft requirements that may conflict with closed-source commercial use. The README does not detail other specific limitations or known issues.