LiteLLM provides a unified Python SDK and proxy server for interacting with over 100 large language model (LLM) APIs, abstracting away provider-specific differences. It targets developers and researchers needing to integrate multiple LLMs seamlessly, offering a consistent OpenAI-like API format for completion, embedding, and image generation calls, along with features like retries, fallbacks, and rate limiting.

How It Works

LiteLLM acts as a translation layer, converting incoming requests into the format required by the target LLM provider. It supports a wide array of providers including OpenAI, Azure, Bedrock, VertexAI, and HuggingFace. The core advantage is the consistent output format and the ability to switch between models and providers with minimal code changes, simplifying complex LLM orchestration.

Quick Start & Requirements

• Install: pip install litellm
• Dependencies: Python 3.8+, openai>=1.0.0, pydantic>=2.0.0. API keys for desired LLM providers are required.
• Docs: https://docs.litellm.ai/
• Deploy: Render https://render.com/deploy?repo=https://github.com/BerriAI/litellm, Railway https://railway.app/template/HLP0Ub?referralCode=jch2ME

Highlighted Details

• Supports 100+ LLM providers for completion, streaming, async calls, and embeddings.
• Offers a proxy server with features like authentication hooks, logging, cost tracking, and rate limiting.
• Integrates with observability tools like Lunary, MLflow, Langfuse, and Helicone via callbacks.
• Provides key management for the proxy server, enabling budget and rate limit configuration per project.

Maintenance & Community

• Active development with frequent updates.
• Community support via Discord.
• Enterprise tier available for professional support and custom integrations.

Licensing & Compatibility

• Primarily licensed under MIT.
• Enterprise features may have different licensing terms.

Limitations & Caveats

• Version 1.0.0+ requires openai>=1.0.0.
• Some providers may have specific limitations or require custom configurations not fully detailed in the main README.