Cortex.cpp is a local AI API platform designed for robotics and AI applications, enabling users to run various AI models (vision, speech, language, tabular, action) without cloud dependency. It targets developers and researchers seeking an efficient, self-contained AI inference solution.

How It Works

Cortex.cpp utilizes a multi-engine architecture, initially supporting llama.cpp, with extensibility for custom engines. It features automatic hardware acceleration for NVIDIA, AMD, and Intel GPUs, optimizing inference performance. The platform exposes an OpenAI-compatible API, simplifying integration with existing tools and workflows.

Quick Start & Requirements

• Installation: Download platform-specific installers (exe, pkg, deb) or use curl -s https://raw.githubusercontent.com/menloresearch/cortex/main/engine/templates/linux/install.sh | sudo bash for other Linux distributions.
• Prerequisites: None explicitly stated beyond OS compatibility.
• Usage: Start server with cortex start, pull models with cortex pull <model_name>, and run models with cortex run <model_name>.
• Docs: Cortex Docs, API Reference

Highlighted Details

• Supports multiple AI model types including Llama 3, Phi-4, Mistral, and Gemma.
• Offers automatic GPU detection and hardware optimization.
• Provides an OpenAI-compatible API for seamless integration.
• Includes nightly builds with advanced features like multiple quantizations and engine management.

Maintenance & Community

• Actively under development with rapid improvements.
• Community support available via Discord.
• Issues can be reported on GitHub.

Licensing & Compatibility

• License not explicitly stated in the README. Compatibility for commercial or closed-source use is not specified.

Limitations & Caveats

The project is under active development, indicating potential for rapid changes and breaking updates. Specific licensing details for commercial use are not provided.