This project provides a unified API gateway for managing multiple Large Language Model (LLM) services, abstracting away provider-specific differences and offering features like load balancing and OpenAI-compatible endpoints. It's designed for users who need to integrate various LLM backends without complex frontend interfaces, benefiting from a single point of access and flexible configuration.

How It Works

Uni-API acts as a reverse proxy, accepting requests in OpenAI format and routing them to configured backend LLM providers. It uniformly converts responses to the OpenAI format. Key features include sophisticated load balancing strategies (channel-level weighted, regional, sequential, round-robin), automatic retries, channel cooling mechanisms for failed providers, fine-grained model timeout settings, and permission controls.

Quick Start & Requirements

• Install/Run: Primarily via Docker.
  • docker run --user root -p 8001:8000 --name uni-api -dit -v ./api.yaml:/home/api.yaml yym68686/uni-api:latest
  • Alternatively, use CONFIG_URL environment variable for remote configuration.
• Prerequisites: Docker, a configuration file (api.yaml).
• Setup: Configuration file creation is the main step.
• Docs: https://github.com/yym68686/uni-api

Highlighted Details

• Supports a wide array of providers including OpenAI, Anthropic, Gemini, Vertex AI, Azure, AWS, Cohere, Groq, Cloudflare, and OpenRouter.
• Features native function calling and image recognition API support for multiple providers.
• Offers granular control over rate limiting, model timeouts, and channel cooldown periods.
• Includes OpenAI moderation capabilities for message content.

Maintenance & Community

• Actively maintained with regular updates and bug fixes.
• Community support channels are not explicitly mentioned, but sponsorship options are available.

Licensing & Compatibility

• Licensed under GPL-3.0.
• The GPL-3.0 license is copyleft, requiring derivative works to also be open-sourced under the same license, which may restrict commercial use or linking with closed-source applications.

Limitations & Caveats

• The GPL-3.0 license imposes significant restrictions on commercial use and integration into proprietary software.
• Configuration is entirely file-based, requiring manual setup for each provider and model.