This library provides React Native bindings for llama.cpp, enabling on-device large language model inference within mobile applications. It targets React Native developers seeking to integrate powerful AI capabilities directly into their iOS and Android apps without relying on external APIs.

How It Works

The library wraps the C++ llama.cpp inference engine, exposing its functionality through a JavaScript interface. It leverages pre-built binaries for iOS and Android, with options to build from source. The design mirrors llama.cpp's server examples, mapping API endpoints to methods like completion, tokenize, and embedding on a LlamaContext object. It supports advanced features like tool calling via Jinja templating and grammar-based sampling using GBNF.

Quick Start & Requirements

• Install: npm install llama.rn
• iOS: Run npx pod-install. Set RNLLAMA_BUILD_FROM_SOURCE=1 in Podfile to build from source. Extended Virtual Addressing recommended. Metal support requires Apple7 GPU or higher; iOS simulator is not supported.
• Android: Add proguard rule if enabled. Set rnllamaBuildFromSource=true in gradle.properties to build from source. Currently supports arm64-v8a/x86_64; 64-bit platforms recommended. No integrated GPU backend.
• Models: Download GGUF format models from HuggingFace.

Highlighted Details

• Supports chat and text completion with partial result callbacks.
• Implements universal tool calling via Jinja templating and chat.cpp.
• Enables output control with GBNF grammar, including JSON schema conversion.
• Provides session loading/saving and completion stopping.
• Includes a Jest mock for testing.

Maintenance & Community

• Built and maintained by BRICKS.
• Apps using llama.rn: BRICKS, ChatterUI, PocketPal AI.
• Related project: llama.node (Node.js binding with similar API).

Licensing & Compatibility

• License: MIT.
• Compatible with commercial and closed-source applications.

Limitations & Caveats

iOS Metal support has minimum GPU requirements and is not available on the simulator. Android is limited to specific architectures and lacks GPU acceleration.