ChatterUI is a native mobile application designed for interacting with Large Language Models (LLMs). It offers a user-friendly interface for both on-device inference using llama.cpp and connecting to various commercial and open-source LLM APIs, catering to mobile users seeking flexible LLM access and chat customization.

How It Works

ChatterUI leverages llama.cpp for on-device LLM execution, utilizing a custom React Native adapter (cui-llama.rn) for integration. This approach allows users to run quantized GGUF models directly on their mobile devices, offering a private and efficient LLM experience. For remote access, it supports numerous APIs including OpenAI, Claude, Cohere, Ollama, and text-generation-webui, with a flexible template system for custom API integrations.

Quick Start & Requirements

• Installation: Download the latest APK from the releases page.
• Prerequisites: Android 12+ recommended. For on-device inference, GGUF models are required. Snapdragon 8 Gen 1+ or Exynos 2200+ recommended for optimized performance with Q4_0 quantization.
• Development: Requires Node.js, Java 17/21 SDK, Android SDK, and Expo.
• Links: Releases, Character Card v2, Custom API Templates

Highlighted Details

• Supports on-device LLM inference via llama.cpp with GGUF models.
• Connects to a wide range of LLM APIs including OpenAI, Claude, Ollama, and more.
• Enables chat with characters supporting the Character Card v2 specification.
• Offers customization for sampler fields and instruct formatting.

Maintenance & Community

• Project is actively maintained by Vali-98.
• Acknowledgements include llama.cpp and cui-llama.rn.

Licensing & Compatibility

• The project appears to be under the MIT License, allowing for commercial use and integration with closed-source applications.

Limitations & Caveats

iOS support is currently unavailable due to a lack of development hardware. Building for Android requires a Linux environment for EAS builds.