<2-3 sentences summarising what the project addresses and solves, the target audience, and the benefit.> TinyRouter addresses the challenge of efficiently utilizing multiple open-source Large Language Models (LLMs) by implementing a lightweight coordinator. This system intelligently routes user queries to the most suitable LLM and assigns it a specific role (Thinker, Worker, Verifier), optimizing performance and cost for diverse tasks. It targets engineers and researchers seeking to maximize the utility of specialized LLM capabilities.

How It Works

The core of TinyRouter is a compact coordinator, comprising a frozen 0.6B encoder and a ~10K-parameter routing head. The encoder transforms input questions into a single vector, which the head then uses to select an LLM from a pool (deepseek-v4-pro, glm-5p2, kimi-k2p6 via Fireworks AI) and assign a role. This process repeats for up to five turns, with a Verifier role capable of early termination. Training employs a derivative-free evolution strategy (separable CMA-ES) optimizing for a binary right/wrong reward signal derived from an automatic grader. This approach prioritizes a minimal, cost-effective routing mechanism over complex orchestration.

Quick Start & Requirements

Installation details are not explicitly provided. Running the coordinator requires significant GPU resources, with an NVIDIA H200 mentioned for its operation. Access to LLMs is facilitated through Fireworks AI, potentially requiring API keys. The project builds upon concepts from TRINITY (arXiv:2512.04695).

Highlighted Details

• Achieves an average score of 0.858 across math and MMLU tasks, outperforming any single model baseline (0.835).
• Demonstrates significant routing benefits on tasks where models exhibit distinct specializations (e.g., knowledge vs. math).
• Core replication and evaluation costs were minimal, approximately $20.89, with training experiments costing $27.22.
• Employs evolutionary computation (sep-CMA-ES) for training the routing head, a novel approach for LLM orchestration.

Maintenance & Community

The repository is maintained by "harrrshall." No specific community channels (e.g., Discord, Slack), roadmap, or notable contributor information are detailed in the provided README.

Licensing & Compatibility

The license type is not specified in the README. Compatibility is designed for open-source LLMs accessible via Fireworks AI.

Limitations & Caveats

Significant GPU hardware (NVIDIA H200) is indicated for running the coordinator. Two implemented upgrades (supervised warm-start and shaped fitness) aimed at improving performance on math tasks have unproven impacts on held-out scores due to experimental noise. The repository's license remains unspecified, posing a potential adoption blocker.