An open-source, OpenAI-compatible LLM gateway, Otari addresses the need for a unified, self-hosted endpoint to manage access across numerous LLM providers. It targets developers and organizations seeking centralized control over API keys, cost management, and usage tracking for their AI applications, offering a flexible alternative to direct provider integration.

How It Works

Otari functions as a proxy server, presenting a single OpenAI-compatible API endpoint. It intercepts requests, authenticates clients using virtual keys, enforces pre-call budgets, resolves provider credentials, and routes requests to over 40 LLM providers via the any-llm library. All interactions, including usage and spend, are logged centrally, providing a consolidated view of AI resource consumption.

Quick Start & Requirements

A metered gateway can be operational in approximately one minute using Docker.

• Primary install/run command: docker run --rm -p 8000:8000 -e OTARI_MASTER_KEY=<your_master_key> -e <PROVIDER>_API_KEY=<your_provider_key> -e OTARI_CONFIG_YAML='default_pricing: true' mzdotai/otari:latest otari serve
• Prerequisites: Docker, an API key for at least one LLM provider (e.g., OpenAI).
• Links: Docs, otari.ai, Launch blog, Discord.

Highlighted Details

• Supports over 40 LLM providers through a single OpenAI-compatible endpoint.
• Enables management of virtual API keys, budgets (per-user/per-key), and detailed usage tracking.
• Features built-in tools like otari_code_execution (sandboxed Python REPL) and otari_web_search.
• Includes request-level guardrails, such as prompt injection blocking, that operate before provider calls.
• Offers both standalone (self-hosted) and hybrid (connected to otari.ai platform) operational modes.

Maintenance & Community

The project is hosted under the mozilla-ai GitHub organization. Community interaction is facilitated via a Discord server.

Licensing & Compatibility

Otari is licensed under the Apache 2.0 license, which permits commercial use and integration into closed-source applications.

Limitations & Caveats

The default otari_web_search backend (SearXNG) is rate-limited for sustained use; production environments require configuration with a licensed search backend. Advanced features like built-in tools and guardrails require specific Docker Compose profiles to be activated.