This toolkit provides production-ready, privacy-first, on-device AI SDKs for iOS and Android applications. It enables developers to integrate powerful language models directly into their apps, offering automatic optimization for performance, privacy, and user experience, thereby enhancing mobile AI capabilities without relying solely on cloud services.

How It Works

The core approach centers on privacy-first, on-device AI execution. The SDKs automatically optimize AI model performance, privacy, and user experience. They support a wide array of frameworks including GGUF (via llama.cpp), Apple Foundation Models, WhisperKit, Core ML, MLX, and TensorFlow Lite on iOS, and GGUF via llama.cpp on Android. Processing occurs locally by default, with intelligent routing capabilities planned for hybrid cloud execution.

Quick Start & Requirements

• Installation:
  • iOS: Swift Package Manager (via Xcode or Package.swift). Core SDK: RunAnywhere. Optional: LLMSwift, WhisperKitTranscription, FluidAudioDiarization.
  • Android: Gradle (RunAnywhereKotlinSDK-android, runanywhere-llm-llamacpp-android) or Maven. JVM target available.
• Prerequisites:
  • iOS: iOS 16.0+, macOS 12.0+, tvOS 14.0+, watchOS 7.0+. Xcode 15.0+, Swift 5.9+. Recommended: iOS 17.0+.
  • Android: Minimum SDK 24 (Android 7.0). Kotlin 2.1.21+, Gradle 8.11.1+, Java 17.
• Links: Full iOS documentation, Full Android documentation, Demo Apps.

Highlighted Details

• Privacy-First, On-Device AI: All processing happens locally by default, prioritizing user data privacy.
• Comprehensive Voice AI Pipeline (iOS): Integrates Voice Activity Detection (VAD), Speech-to-Text (STT), Large Language Model (LLM), and Text-to-Speech (TTS) components.
• Structured Outputs: Facilitates type-safe JSON generation with schema validation using the Generatable protocol.
• Multi-Framework Support: Integrates with llama.cpp (GGUF/GGML), Apple Foundation Models, Core ML, MLX, and TensorFlow Lite.
• Performance Analytics: Provides real-time metrics including tokens per second, time to first token, latency, and memory usage.
• Model Management: Features automatic model discovery, downloading with progress tracking, and lifecycle management.

Maintenance & Community

The project is actively developed with a roadmap including Android SDK feature parity and hybrid routing in the next release. Upcoming features include remote configuration and extended model support. Community support is available via Discord, GitHub Issues, email (founders@runanywhere.ai), and Twitter (@RunanywhereAI).

Licensing & Compatibility

The project is licensed under the Apache License 2.0, which is permissive for commercial use and integration into closed-source applications. Third-party components include llama.cpp (MIT License) and MLC-LLM (Apache License 2.0).

Limitations & Caveats

The Android SDK is noted to be working towards full feature parity with the iOS SDK. Hybrid on-device and cloud execution is a planned feature for a future release. Extended model format support is also listed as an upcoming feature. The versioning suggests ongoing development, and users should evaluate stability for production use cases.