Wayfinder Router is a CLI tool designed to deterministically route queries between local and hosted Large Language Models (LLMs). It addresses the challenge of optimizing LLM costs and latency by intelligently directing prompts to the most appropriate model based on a calculated complexity score. This enables users to send simple requests to inexpensive local models and complex ones to powerful, costly hosted services, thereby reducing operational expenses without sacrificing performance for demanding tasks. The primary audience includes engineers, researchers, and power users seeking efficient LLM orchestration.

How It Works

Wayfinder Router employs a deterministic, offline scoring mechanism that analyzes a prompt's structure (e.g., length, headings, code blocks) and wording (e.g., presence of proofs, mathematical terms) to generate a complexity score between 0.0 and 1.0. This score is calculated without making any actual LLM calls, ensuring sub-millisecond decision times and complete offline operation. This approach contrasts with other routers that rely on model inferences or API calls for routing decisions, thereby avoiding added latency, cost, and randomness. The system is designed for flexibility, allowing users to configure tiered routing, chain multiple routers, or calibrate the scoring on their own data for personalized optimization.

Quick Start & Requirements

• Primary install: pip install wayfinder-router
• Gateway install: pip install "wayfinder-router[gateway]"
• UI install: pip install "wayfinder-router[ui]"
• Prerequisites: Python environment. OpenAI-compatible API endpoints are required for actual LLM responses. API keys are managed via environment variables or optional secret store integrations.
• Demo: Terminal chat: uvx wayfinder-router chat --dry-run or pip install wayfinder-router && wayfinder-router chat. Web chat: pip install "wayfinder-router[gateway]" && wayfinder-router webchat --dry-run.
• Documentation: Links to "Quickstart", "Benchmark", "How it compares", "Explainer", and "Changelog" are mentioned in the README.

Highlighted Details

• Deterministic, sub-millisecond, and fully offline routing decisions.
• Supports calibration on custom data to tune routing thresholds and weights.
• Compatible with any OpenAI-compatible API endpoint, including popular providers and local servers like Ollama, vLLM, and LM Studio.
• API keys are securely handled by reading from environment variables at request time, never stored in configuration files.
• Offers tiered routing configurations and supports composition with other routing mechanisms.
• Includes an init command to scaffold configuration files and a doctor command to verify setup and key resolution.
• Provides a serve --dry-run option for immediate feedback on routing decisions without requiring backend models.

Maintenance & Community

The repository is maintained by itsthelore. No specific details regarding community channels (like Discord/Slack), notable contributors, sponsorships, or a public roadmap were found in the provided text.

Licensing & Compatibility

The license type is not explicitly stated in the provided README content. This omission requires further investigation for commercial use or closed-source integration considerations. The tool is designed to be compatible with any OpenAI-compatible API.

Limitations & Caveats

The default routing relies primarily on prompt structure; lexical cues (e.g., mathematical terms) are opt-in and have shown limited generalization in tests. Wayfinder Router may not perform optimally on prompts that are purely semantic and lack structural indicators. While generally effective, it has been noted to perform comparably to random chance on specific "short-but-hard" items in certain benchmarks, though its deterministic and offline nature remains a key advantage. The absence of a stated license is a significant caveat for adoption.