RESTai provides an open-source AI-as-a-Service (AIaaS) platform for creating and consuming AI projects via a REST API. It targets developers and researchers looking to easily integrate various AI capabilities, including RAG, SQL querying, agents, and image generation, into their applications. The platform offers a unified API layer over diverse LLMs and embedding models, simplifying complex AI workflows.

How It Works

RESTai leverages LlamaIndex and Langchain as core orchestration frameworks, enabling support for a wide array of LLMs (public and local via Ollama/vLLM) and embedding models. Its architecture is designed to be stateless for production, with optional stateful modes for development. Projects can be dynamically configured for different tasks like RAG (with sandboxing and evaluation via deepeval), RAGSQL (auto-translating natural language to SQL), agents (with pluggable tools), and vision models (img2text, text2img, img2img).

Quick Start & Requirements

• Install and run locally: make dev
• Production deployment: make install && make start or via Docker (docker compose --env-file .env up --build).
• Supports optional profiles for Redis, MySQL, and PostgreSQL.
• API documentation available at /swagger.
• Frontend source code: https://github.com/apocas/restai-frontend

Highlighted Details

• Supports multiple project types: RAG, RAGSQL, inference, vision, router, agent.
• Dynamic loading of new LLM, embedding, vectorstore, loader, tool, and image generators.
• Built-in evaluation for RAG projects using deepeval.
• OpenAI-compatible proxy management via LiteLLM.
• RAG projects offer a "sandboxed" mode for default answers when no embeddings are found.

Maintenance & Community

• Developed by Pedro Dias (@pedromdias).
• Community links are not explicitly mentioned in the README.

Licensing & Compatibility

• Licensed under the Apache License, Version 2.0.
• Permissive license suitable for commercial use and integration with closed-source applications.

Limitations & Caveats

The README does not detail specific hardware requirements (e.g., GPU) for running local models, nor does it mention performance benchmarks or known limitations of the various project types. Community support channels are not specified.