A Cloudflare Worker-based reverse proxy designed to unify access to various Large Language Model (LLM) APIs, including OpenAI, Gemini, and Groq. It presents a single, OpenAI-compatible API endpoint, allowing developers to integrate multiple LLM providers seamlessly using familiar SDKs and tools, simplifying LLM orchestration and potentially optimizing costs or performance.

How It Works

The project leverages Cloudflare Workers to create a serverless API gateway. Users deploy the provided JavaScript code (api/llm_api_proxy.js) to their own Cloudflare Worker instance. This worker intercepts API requests and routes them to the appropriate LLM backend based on the request path (e.g., /v2/gemini). By adhering to the OpenAI API specification, it ensures compatibility with existing client libraries and applications. For certain providers like Groq, an additional request relay layer using Vercel and FastAPI is employed.

Quick Start & Requirements

• Deployment: Deploy the api/llm_api_proxy.js code as a new Cloudflare Worker.
• Prerequisites: A Cloudflare account is required. Configuring a custom domain is recommended for reliable access, particularly within regions where the default workers.dev domain may be restricted (e.g., China).
• Links:
  • Demo API: https://llmapi.ultrasev.com

Highlighted Details

• Provides a unified API interface for OpenAI, Gemini, Groq, and Mistral LLM services.
• Compatible with the OpenAI API specification, enabling direct use with the OpenAI SDK.
• Can be integrated with third-party tools like Chatbox and Immersive Translate.

Maintenance & Community

• The README does not detail specific contributors, sponsorships, or community channels (e.g., Discord, Slack).
• A changelog is referenced for updates.

Licensing & Compatibility

• The license type is not explicitly stated in the provided README content.
• Compatibility for commercial use or integration within closed-source projects is undetermined without a specified license.

Limitations & Caveats

• The default workers.dev domain may face accessibility issues in certain geographic locations, necessitating the use of a custom domain.
• Streaming output for LLM responses is not currently supported.
• Accessing the Gemini API through the proxy can be subject to Google AI's regional IP restrictions, potentially requiring the use of specific proxy regions (e.g., US) or domestic IPs.
• The Groq proxy implementation includes an intermediate relay layer (Vercel + FastAPI), which may be a consideration for users prioritizing direct API connections.