Summary

This project aggregates free-tier API access from ~14 AI providers into a single, OpenAI-compatible endpoint. It targets individual developers and researchers seeking to experiment with a vast token pool (~1.3B/month) without the complexity of managing multiple SDKs, rate limits, and keys, offering a unified interface for personal experimentation.

How It Works

FreeLLMAPI functions as a smart proxy, routing requests to various LLM providers based on availability and configured priority. Its core logic includes automatic failover to the next provider upon encountering rate limits (429) or server errors (5xx), per-key usage tracking (RPM, RPD, TPM, TPD) to respect free-tier caps, and encrypted storage of API keys. It supports streaming, non-streaming, and tool-calling requests, maintaining conversation context with sticky sessions.

Quick Start & Requirements

• Primary install/run command: npm run dev (development), node server/dist/index.js (production).
• Prerequisites: Node.js 20+, npm.
• Setup: Requires generating an encryption key via .env.example.
• Links: Vite dev UI at http://localhost:5173, production server at :3001.

Highlighted Details

• OpenAI-Compatible Endpoint: Offers a /v1/chat/completions endpoint compatible with official OpenAI SDKs and clients.
• Intelligent Failover & Usage Tracking: Automatically retries requests on different providers and enforces per-key rate limits.
• Encrypted Key Storage: API keys are AES-256-GCM encrypted in SQLite and decrypted in-memory.
• Admin Dashboard: Provides a React/Vite UI for key management, fallback chain configuration, and analytics.
• Resource Efficient: Deployable on low-power hardware like a Raspberry Pi 4 with minimal RAM usage.
• Tool Calling: Supports OpenAI-style tool use and function calling across providers.

Maintenance & Community

• Contributors: Acknowledges @moaaz12-web for tool-calling support.
• Community: "Issues and discussions are open," "PRs Welcome." No specific community links (Discord, Slack) are provided in the README.

Licensing & Compatibility

• License: MIT License.
• Compatibility: Strictly for "personal experimentation only" and "single-user by design." Certain providers' free tiers (e.g., NVIDIA, GitHub) are marked for "evaluation" or "experimentation" only, and Cohere's trial ToS explicitly forbids personal use. Public exposure or multi-tenant use is discouraged.

Limitations & Caveats

• Model Capabilities: Does not provide access to frontier models; free tiers are limited to mid-tier capabilities.
• Performance Variability: Model intelligence and latency can degrade as daily token caps are reached, especially later in the day.
• Free Tier Instability: Providers may alter free tier terms without notice, potentially breaking functionality.
• No Production Guarantees: Lacks an SLA and is not intended for production use; reliability relies on upstream providers.
• Unsupported Features: Excludes embeddings, image/audio generation, vision, legacy completions, moderation, and batch requests (n > 1).