This project provides fully offline, on-device multimodal chat capabilities for iOS, Android, and HarmonyOS NEXT, enabling users to run MiniCPM-V family models directly on their mobile devices. It targets developers and power users seeking to integrate advanced AI features into mobile applications without relying on cloud infrastructure, offering enhanced privacy and potentially lower latency.

How It Works

The core of the project utilizes a fork of llama.cpp specifically adapted for MiniCPM-V models. This enables efficient execution of large multimodal language models directly on mobile hardware. The approach bypasses the need for server-side processing, allowing for complete local inference and offering a novel solution for deploying sophisticated AI on resource-constrained mobile platforms.

Quick Start & Requirements

• Primary Install/Run: Clone the repository with submodules (git clone --recurse-submodules --shallow-submodules). Platform-specific builds require native IDEs: Xcode for iOS (after building llama.xcframework via ./scripts/build_xcframework.sh), Android Studio for Android (using ./gradlew assembleDebug), and DevEco Studio for HarmonyOS. Pre-built packages are available via DOWNLOAD.md.
• Prerequisites:
  • iOS: Xcode, Command Line Tools, CMake (≥3.28). Apple Developer account may be needed for device deployment.
  • Android: Android Studio (Giraffe+), Android SDK/NDK (pinned to 27.0.12077973), CMake (3.22.1). Requires a 64-bit ARM SoC device.
  • HarmonyOS: DevEco Studio 5.0+, HarmonyOS Native SDK/NDK. Requires API 12+ device/emulator, 64-bit ARM architecture.
  • Hardware: Device RAM requirements vary by model (e.g., ≥8 GB recommended for MiniCPM-V 2.6).
• Links: Official GGUF model files are available on HuggingFace and ModelScope (links provided in README).

Highlighted Details

• Supports three MiniCPM-V model versions (2.6, 4.0, 4.6) and a text-only MiniCPM5-1B.
• Fully offline, on-device execution via llama.cpp integration.
• Platform-specific demos provided for iOS (Xcode), Android (Gradle/Kotlin), and HarmonyOS NEXT (DevEco Studio/ArkTS).
• Detailed hardware requirements and model file sizes (GGUF) are specified for each model version.

Licensing & Compatibility

The license for the OpenBMB/MiniCPM-V-Apps repository itself is not explicitly stated in the provided README. It bundles llama.cpp, which is typically MIT licensed. Compatibility for commercial use or linking with closed-source applications would depend on the specific licenses of the app repository and the MiniCPM-V model weights.

Limitations & Caveats

Building the llama.xcframework for iOS may produce harmless warnings. Deployment to iOS devices might require an Apple Developer account. Specific NDK and CMake versions are pinned for Android builds. Device RAM requirements are critical; lower-spec devices may struggle, especially with larger models like V 2.6, and larger context windows increase RAM usage linearly. The project manages llama.cpp as a submodule, necessitating specific cloning and update procedures.