UncommonRoute addresses the significant cost of using premium Large Language Models (LLMs) for tasks that do not require their full capabilities. It provides an automatic LLM router that functions as a drop-in OpenAI proxy, intelligently selecting the most cost-effective model capable of fulfilling a given request. This solution offers substantial savings (82%) while maintaining high accuracy (79.4%) and task pass rates (93.4%), benefiting developers and organizations looking to optimize LLM operational expenses.

How It Works

The core of UncommonRoute is its sophisticated routing engine, which analyzes incoming requests using three independent signals: Metadata (examining conversation structure and tool usage), Embedding (comparing semantic similarity to known task patterns via bge-small), and Structural (evaluating text complexity). These signals contribute to an ensemble decision that identifies the appropriate capability tier for the task. The router then selects the cheapest model within that tier, defaulting to a more expensive option if uncertain to ensure task completion. The system learns over time, adjusting signal weights and growing its embedding index. The current v2 architecture was rebuilt from scratch to address accuracy issues encountered with v1 on complex, real-world agent conversations, achieving a 78% tier match accuracy. End-to-end routing overhead is approximately 20-25ms on a warm process.

Quick Start & Requirements

Installation is straightforward via pip: pip install uncommon-route. Users can connect upstream providers either through Commonstack's managed service (requiring UNCOMMON_ROUTE_UPSTREAM and UNCOMMON_ROUTE_API_KEY environment variables, offering consolidated billing across multiple LLM providers) or by bringing their own API keys (uncommon-route provider add <name> <api-key>). Client applications are then pointed to the local proxy (defaulting to http://localhost:8403), with specific configurations for OpenAI SDK (/v1 endpoint) and Anthropic (/ endpoint). Model requests should target uncommon-route/auto.

Highlighted Details

• Achieves 82% cost savings compared to always using premium models.
• Reports a 93.4% task pass rate and 78% accuracy in tier matching.
• Routing overhead is approximately 20–25ms for warm processes.
• Includes a web-based dashboard (http://localhost:8403/dashboard/) for real-time monitoring, interactive playground, cost tracking, and configuration.
• Supports multiple routing modes: auto (balanced), fast (cost-first), and best (quality-first).

Maintenance & Community

The project is maintained by CommonstackAI. The provided README does not detail specific community channels (like Discord or Slack), notable contributors, or sponsorships.

Licensing & Compatibility

UncommonRoute is released under the MIT License. This license is highly permissive, allowing for commercial use, modification, and distribution, including integration within closed-source applications without significant copyleft restrictions.

Limitations & Caveats

The router incurs a cold start overhead of a few hundred milliseconds for the first request. While the v2 architecture represents a significant improvement, the project's history indicates that achieving high accuracy on diverse, real-world agent conversations is complex and may require ongoing refinement.