Summary

Octopus addresses the need for unified management and efficient access to multiple Large Language Model (LLM) APIs for individual users. It aggregates various LLM provider channels, offering load balancing, protocol conversion, and cost tracking through an elegant web UI. This service benefits users by simplifying API integration, ensuring service stability, and providing insights into LLM usage costs.

How It Works

The service functions as an LLM API gateway, connecting to multiple LLM providers as "channels." These channels can be organized into "groups," allowing unified access under a single model name. Octopus supports several load balancing strategies (Round Robin, Random, Failover, Weighted) to distribute requests efficiently across configured channels. It also performs seamless protocol conversion between OpenAI Chat, OpenAI Responses, and Anthropic API formats, simplifying integration with diverse LLM backends. Model pricing and availability are automatically synchronized from external sources like models.dev.

Quick Start & Requirements

• Primary Install/Run: Docker (docker run -d --name octopus -v /path/to/data:/app/data -p 8080:8080 bestrui/octopus), Docker Compose (wget https://raw.githubusercontent.com/bestrui/octopus/refs/heads/dev/docker-compose.yml && docker compose up -d), or download pre-compiled binaries from Releases.
• Prerequisites: For building from source: Go 1.24.4, Node.js 18+, npm or pnpm.
• Default Credentials: Username: admin, Password: admin (immediate change recommended).
• Links: GitHub repository (implied), Docker Hub (implied by image name).

Highlighted Details

• Multi-Channel Aggregation: Integrates numerous LLM providers under a unified interface.
• Load Balancing: Offers Round Robin, Random, Failover, and Weighted distribution modes.
• Protocol Conversion: Supports seamless conversion between OpenAI Chat, OpenAI Responses, and Anthropic API formats.
• Automatic Sync: Periodically synchronizes model pricing data and available model lists from models.dev.
• Analytics: Provides comprehensive statistics on requests, token consumption, and costs.
• Elegant UI: Features a clean and user-friendly web management panel.

Maintenance & Community

The project acknowledges contributions from looplj/axonhub for its LLM API adaptation module and sst/models.dev for AI model pricing data. No specific community channels (like Discord/Slack) or details on active maintainers/sponsorships are provided in the README.

Licensing & Compatibility

The provided README text does not specify the software license. Users should verify licensing terms before adoption, especially for commercial use.

Limitations & Caveats

Proper shutdown procedures (e.g., Ctrl+C, SIGTERM) are critical to prevent data loss for in-memory statistics; forced termination (kill -9) is strongly discouraged. Default administrative credentials require immediate modification for security. Building from source involves a multi-step process including frontend compilation.