Jan is an open-source, offline AI assistant designed as a privacy-focused alternative to services like ChatGPT. It targets users who want local control over their AI interactions, offering a user-friendly interface for downloading and running various Large Language Models (LLMs) on personal hardware.

How It Works

Jan is powered by Cortex.cpp, an embeddable C++ AI engine that acts as a backend. Cortex.cpp supports multiple inference engines, including llama.cpp (default), ONNX, and TensorRT-LLM, providing flexibility across different hardware architectures. This multi-engine approach aims to optimize performance and compatibility for a wide range of devices, from standard PCs to multi-GPU setups.

Quick Start & Requirements

• Installation: Download pre-built binaries for Windows (.exe), macOS (.dmg), or Linux (.deb, .AppImage) from https://jan.ai/ or GitHub Releases.
• Prerequisites:
  • macOS: 13+
  • Windows: 10+
  • Linux: glibc 2.27+, gcc/g++/cpp 11+
  • GPU Support: NVIDIA GPU with CUDA Toolkit 11.7+ and driver 470.63.01+ required for NVIDIA GPU acceleration. Apple M-series chips are also supported.
• Development Setup: Requires Node.js >= 20.0.0, Yarn >= 1.22.0, and Make >= 3.81. Run make dev for development.
• Documentation: https://jan.ai/docs

Highlighted Details

• Supports popular LLMs like Llama, Gemma, Mistral, and Qwen.
• Features a Model Library for easy LLM management.
• Includes a local API server with OpenAI-equivalent endpoints.
• Offers extensibility for customizability.

Maintenance & Community

• Actively in development with frequent updates.
• Community support available via Discord: https://discord.gg/janai
• Bug reports and feature requests via GitHub issues.

Licensing & Compatibility

• Licensed under AGPLv3. This is a strong copyleft license, requiring derivative works to also be open-sourced under AGPLv3.

Limitations & Caveats

Jan is explicitly stated to be in development, warning of potential breaking changes and bugs. The AGPLv3 license may impose significant restrictions on integration into closed-source commercial products.