A self-hosted, OpenAI-compatible LLM router, OrcaRouter Lite addresses the need for efficient and cost-effective LLM API management. It targets developers and power users by offering automatic model selection based on capability and cost, a managed hosted fallback, and local analytics, simplifying integration and optimizing spend across various LLM providers.

How It Works

OrcaRouter Lite operates as a local server proxying LLM requests. Its key innovation is model="auto", which intelligently routes requests to the cheapest model capable of fulfilling the user's requirements (e.g., tools, vision, JSON mode) across configured providers. It supports streaming responses and provides a built-in dashboard for configuration and analytics, functioning without mandatory external databases like Postgres or Redis.

Quick Start & Requirements

• Self-hosted (Path A): Clone the repository, copy .env.example to .env, add provider API keys (e.g., OPENAI_API_KEY), and run docker compose up. The API base URL is http://localhost:8000/v1.
• Hosted (Path B): Register at https://www.orcarouter.ai, obtain an API key, and use https://api.orcarouter.ai/v1 as the base URL.
• Prerequisites: Docker is required for the self-hosted option.
• Documentation: Further details are available at docs.orcarouter.ai/introduction.

Highlighted Details

• model="auto" Routing: Dynamically selects the cheapest capable LLM provider based on real-time pricing and feature requirements, eliminating manual routing logic.
• OpenAI Compatibility: Seamlessly integrates with any tool or SDK designed for OpenAI's Chat Completions API.
• Streaming Support: Provides drop-in compatibility for streaming Server-Sent Events (SSE) responses.
• BYOK & Hosted Fallback: Supports Bring Your Own Keys for local providers and integrates with api.orcarouter.ai as a managed upstream provider.
• Cross-Provider Prompt Caching: Caches deterministic requests across different LLM providers for zero-cost, instant retrieval.
• Local Analytics Dashboard: Offers insights into spend, latency, savings, and model reachability without sending telemetry.
• Extensive Model Catalog: Dynamically loads and manages over 100 chat models from LiteLLM's pricing database.

Maintenance & Community

The project has a clear roadmap with many features already implemented. No specific community channels (e.g., Discord, Slack) are listed in the README.

Licensing & Compatibility

The project is released under the MIT License, permitting commercial use and integration within closed-source applications.

Limitations & Caveats

This "single-workspace edition" deliberately omits features such as multi-tenancy, role-based access control (RBAC), single sign-on (SSO), billing management, and advanced administrative consoles. These capabilities are reserved for the hosted product or a forthcoming Teams edition.