Dalai provides a simplified interface for running LLaMA and Alpaca large language models locally. It targets developers and power users seeking an easy-to-use, cross-platform solution for local LLM inference, offering a web UI and a JavaScript API.

How It Works

Dalai leverages llama.cpp and alpaca.cpp for efficient model execution, abstracting away complex setup. It manages model downloads and provides a Socket.IO-based API for remote interaction, alongside a direct Node.js API for local execution. This approach aims for broad compatibility and ease of use across Linux, macOS, and Windows.

Quick Start & Requirements

• Docker: docker compose build && docker compose run dalai npx dalai alpaca install 7B && docker compose up -d
• macOS/Windows: Requires Node.js >= 18. macOS may need cmake and pkg-config via Homebrew. Windows requires Visual Studio with C++, Node.js, and Python development workloads, and commands must be run in cmd.
• Linux: Requires Python <= 3.10 and Node.js >= 18. May need build-essential, python3-venv (Debian/Ubuntu) or make, automake, gcc, gcc-c++, kernel-devel, python3-virtualenv (Fedora).
• Disk Space: Quantized models range from ~4GB (7B) to ~40GB (65B). Full models require significantly more.
• Memory: 7B models require ~4GB RAM, 13B ~8GB, 30B ~16GB, 65B ~32GB.
• Docs: https://github.com/cocktailpeanut/dalai

Highlighted Details

• Cross-platform support (Linux, macOS, Windows).
• Includes a hackable web application and JavaScript API.
• Supports both LLaMA and Alpaca model families.
• Models are downloaded and managed via llama-dl CDN.

Maintenance & Community

• Active development indicated by recent updates and community channels (GitHub, Twitter, Discord).

Licensing & Compatibility

• The README does not explicitly state a license. Compatibility for commercial or closed-source use is undetermined.

Limitations & Caveats

• Linux support requires Python 3.10 or lower due to potential PyTorch compatibility issues.
• The project's licensing is not specified, which may impact commercial adoption.