SimpleAI provides a self-hosted API for large language models, acting as an alternative to proprietary services. It targets developers and researchers who want to experiment with various AI models, create benchmarks, or handle specific use cases without relying on external providers. The primary benefit is flexibility and independence in AI model deployment and integration.

How It Works

SimpleAI exposes OpenAI-compatible API endpoints (text completion, chat completion, embeddings) via gRPC. This architectural choice decouples the API layer from model inference, allowing for multi-language support and easier integration of diverse models. Users define and expose their models by implementing specific Python interfaces and configuring them in models.toml, pointing to gRPC services that host the actual model inference.

Quick Start & Requirements

• Install: pip install simple_ai_server or from source: pip install git+https://github.com/lhenault/simpleAI
• Requirements: Python 3.9+
• Setup: Run simple_ai init to create models.toml, then simple_ai serve to start the server.
• Docs: Swagger UI (once server is running)

Highlighted Details

• Replicates core OpenAI LLM API endpoints: text completion, chat completion, and embeddings.
• Supports both streaming and non-streaming responses.
• Compatible with OpenAI Python client and other "open" AI clients.
• Allows custom model integration via gRPC services.

Maintenance & Community

• Project is described as "work in progress."
• Open to contributions (PRs, issues, documentation).
• Development environment setup requires make and poetry.

Licensing & Compatibility

• The README does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

Image, audio, file, fine-tune, and moderation endpoints are not yet supported. The project is actively under development, and some features may be experimental or incomplete. Customization for CORS or API versioning requires custom server scripts.