LeapfrogAI is a self-hosted AI platform designed for deployment in air-gapped, cloud-native, and edge environments. It provides a comprehensive suite of tools including a vector database, model backends, an API compatible with OpenAI's, and a user interface, enabling organizations to leverage generative AI without relying on external services, thereby ensuring data independence and cost-effectiveness.

How It Works

LeapfrogAI is built on Defense Unicorns' secure runtime environment (UDS), facilitating deployment via Kubernetes. It offers a monorepo structure with distinct packages for its API, SDK, UI, and various model backends (e.g., llama-cpp-python, vLLM, Whisper). The platform supports Retrieval Augmented Generation (RAG) and integrates with DeepEval for its evaluation framework. Its architecture prioritizes compatibility with existing OpenAI tooling through a similar API structure.

Quick Start & Requirements

The preferred deployment method is a local Kubernetes deployment using UDS. Detailed system requirements and instructions are available on the LeapfrogAI documentation website.

Highlighted Details

• OpenAI-compatible API for seamless integration with existing tools.
• Supports multiple backends including llama-cpp-python, vLLM, and Whisper.
• Includes an evaluation framework integrated with DeepEval.
• Offers "flavors" for deployment using upstream or hardened IronBank images.

Maintenance & Community

The project is supported by a community including Defense Unicorns, Beast Code, Chainguard, Exovera, Hypergiant, Pulze, and various US military branches. The project is currently pausing new feature development to explore other AI capabilities.

Licensing & Compatibility

The repository's licensing is not explicitly stated in the provided README. Compatibility for commercial use or closed-source linking would require clarification on the license.

Limitations & Caveats

The project is currently pausing new feature development. The vLLM backend has a noted limitation requiring a CUDA-enabled PyTorch build for ARM64, which is not readily available via standard package managers. Some component flavors are not yet available as quick-start bundles.