PocketPal AI is a mobile application designed to bring the power of small language models (SLMs) directly to your smartphone for offline, private AI assistance. It targets users who want to leverage AI capabilities without relying on internet connectivity or compromising their data privacy, offering a convenient and secure on-device experience.

How It Works

PocketPal AI utilizes the llama.cpp library, integrated via llama.rn bindings for React Native, to enable efficient on-device inference of various SLMs. This approach allows for direct interaction with models like Phi, Gemma, and Qwen without sending data to external servers. The app manages model loading and offloading automatically to optimize device memory usage.

Quick Start & Requirements

• Installation: Download from the App Store (iOS) or Google Play (Android).
• Development Setup: Requires Node.js (v18+), Yarn, React Native CLI, Xcode (iOS), and Android Studio (Android).
• Hugging Face Integration: Requires a Hugging Face access token for gated models.
• Documentation: https://github.com/a-ghorbani/pocketpal-ai

Highlighted Details

• Offline, on-device AI assistance with no internet required.
• Supports multiple SLMs (Danube, Phi, Gemma, Qwen) with Hugging Face Hub integration.
• Features include message editing, personalized "Pals" with custom personalities, and a benchmarking tool.
• Optimized for iOS and Android, with recent iPad and localization support.

Maintenance & Community

The project is actively developed with recent updates and planned features for 2025. Contributions are welcomed via GitHub.

Licensing & Compatibility

Licensed under the MIT License, permitting commercial use and integration with closed-source applications.

Limitations & Caveats

Text formatting preservation during copying is currently limited. Performance may vary depending on the specific SLM and device hardware.