Summary

Modelrelay is an OpenAI-compatible local router that benchmarks and dynamically selects the best-performing free coding AI models across providers. It offers developers a cost-effective solution by automatically routing requests to the fastest, most capable LLM, eliminating direct API payments and simplifying integration.

How It Works

This project acts as a local API gateway, continuously evaluating free AI coding models from providers like NVIDIA, Groq, and Ollama. Upon receiving a request, modelrelay intelligently selects the optimal backend model based on speed and capability, forwarding the query. Its OpenAI-compatible interface ensures seamless integration with existing applications as a drop-in replacement for direct API calls.

Quick Start & Requirements

Installation is via NPM (npm install -g modelrelay) or Docker. For NPM, run modelrelay post-install; the service is accessible at http://localhost:7352/. Docker users require Docker Engine/Compose, then docker compose up -d --build after fetching repository files. Both methods expose an OpenAI-compatible endpoint at http://127.0.0.1:7352/v1.

Highlighted Details

• Supports 80+ models from 11+ providers (e.g., Kimi K2.5, Minimax M2.5, Deepseek V3.2).
• Features auto-fastest routing and grouped model IDs (e.g., minimax-m2.5) for provider-specific QoS selection.
• Offers seamless integration with OpenClaw and OpenCode via modelrelay onboard.
• Supports streaming and non-streaming API requests.
• Includes an automatic update mechanism for the CLI tool, enabled by default.

Maintenance & Community

Community support, discussions, and feature requests are managed via a dedicated Discord server. The project provides robust CLI commands for updates (modelrelay update, modelrelay autoupdate), configuration management, and service status, indicating active development.

Licensing & Compatibility

The repository's README does not explicitly state a software license. This lack of clear licensing information is a significant adoption blocker, especially for commercial use. Its OpenAI-compatible API facilitates broad compatibility with existing AI tooling.

Limitations & Caveats

The primary limitation is the absence of a declared software license, leaving usage rights ambiguous. The project relies on free tiers from external providers, whose terms are subject to change. The README offers limited insight into known bugs or performance bottlenecks beyond the dynamic routing capabilities.