This project offers an Android application enabling local execution of Large Language Models (LLMs) like Gemma and Qwen directly on mobile devices. It provides offline, private, real-time chat, question answering, and advanced input capabilities (voice, image, OCR, camera), targeting users prioritizing on-device AI processing and data privacy.

How It Works

The application utilizes LiteRT and ONNX Runtime backends for efficient on-device LLM inference. A minimal base APK is installed, with users downloading desired models (Gemma, Qwen) post-installation. This architecture ensures full offline operation after model acquisition, supporting streaming responses, persistent local chat history, and in-app model management (download, switch, delete). It integrates voice input, image analysis via OCR, and camera capture for enhanced user interaction.

Quick Start & Requirements

• Install/Run: Clone the repository, open in Android Studio, build, and install on a physical Android device.
• Prerequisites: Android Studio, a physical Android device (emulators for UI checks), 4GB+ RAM (more recommended for larger models), temporary internet connection for model downloads.
• Links: No direct quick-start or demo links provided.

Highlighted Details

• Fully on-device LLM chat for private, offline use.
• Supports voice input, image input (with OCR), and camera capture for richer prompting.
• In-app model management: download, switch, and delete models to conserve storage.
• Backend support for ONNX (Qwen) and LiteRT (Qwen, Gemma), with LiteRT enabling hardware acceleration.
• Persistent multi-turn chat with local history and markdown-rendered replies.

Maintenance & Community

The provided README does not detail specific contributors, community channels (e.g., Discord, Slack), sponsorships, or a public roadmap.

Licensing & Compatibility

• Gemma 4 models are licensed under Apache License 2.0, permitting responsible commercial use.
• Qwen models adhere to their upstream license terms; users must review these for redistribution or commercial applications.
• The app facilitates local LLM deployment, with model licenses governing commercial compatibility.

Limitations & Caveats

• An internet connection is required solely for initial model downloads.
• Model performance and functionality are contingent upon the specific model build and the target Android device's hardware capabilities.
• Emulators are primarily suitable for UI verification, not comprehensive functional testing.