Flama is a production framework designed to simplify the deployment of predictive and generative AI models. It enables users to expose any model as a production-ready API with a single line of code, offering compatibility with OpenAI, Anthropic, and Ollama endpoints, an integrated chat UI, and native Model Context Protocol (MCP) support. This framework targets engineers and researchers, streamlining the transition from model development to production deployment by providing a unified interface and high-performance serving capabilities.

How It Works

Flama's core innovation lies in its .flm portable artifact format, which packages models from diverse frameworks like scikit-learn, TensorFlow, PyTorch, and LLMs into a standardized structure. This abstraction ensures consistency, allowing any packaged model to be served via HTTP with a unified API. The underlying serving engine leverages a Rust-compiled core for optimized routing, JSON handling, and compression, distributed as pre-compiled native wheels for broad compatibility. Models can also be directly downloaded and packaged from the HuggingFace Hub.

Quick Start & Requirements

• Primary Install: pip install flama
• Optional Extras: Install specific functionalities like schema validation (flama[pydantic]), database integration (flama[database]), or generative AI serving (flama[llm]). Use flama[full] for all extras.
• Prerequisites: Python 3.10 to 3.14. Native wheels are available for Linux, macOS, and Windows, eliminating the need for a Rust toolchain. GPU/CUDA is required for certain LLM backends (e.g., vLLM on Linux).
• Links: Full documentation is available at https://flama.dev/docs/.

Highlighted Details

• Packages models from any mainstream framework (scikit-learn, TensorFlow, PyTorch, LLMs) into a single, portable .flm artifact.
• Serves LLMs with compatible endpoints for OpenAI, Anthropic, and Ollama, alongside native MCP endpoints for AI agent integration.
• Features a built-in, polished streaming chat UI accessible at /chat/, supporting Markdown, LaTeX, and Mermaid rendering.
• Provides native, first-class support for the Model Context Protocol (MCP), automatically deriving JSON schemas from Python type hints to expose tools and resources to AI agents.
• Utilizes a Rust-powered core for high-performance routing, JSON encoding, and request parsing, compiled into native wheels.

Maintenance & Community

The project is authored by José Antonio Perdiguero López (@perdy) and Miguel Durán-Olivencia (@migduroli). Contributions are welcomed via pull requests after reviewing the contributing documentation. Questions and ideas can be discussed in GitHub Discussions. Users are encouraged to star the repository on GitHub to support development.

Licensing & Compatibility

Flama is released under the Apache 2.0 license. This permissive license allows for commercial use and integration into closed-source projects without significant restrictions.

Limitations & Caveats

The LLM serving backend automatically selects between vLLM (Linux) and MLX (Apple Silicon), which may imply platform-specific performance characteristics or feature availability. Advanced features such as schema validation, database integration, and LLM serving are modular and require explicit installation as optional extras.