LiteRT is Google's on-device AI framework for deploying machine learning and generative AI models on edge platforms. It offers efficient model conversion, runtime, and optimization, building on the legacy of TensorFlow Lite with enhanced performance and simplified hardware acceleration for developers targeting mobile and embedded systems.

How It Works

LiteRT V2 (Next) introduces a new API designed for streamlined development, featuring automated accelerator selection, true asynchronous execution, and efficient I/O buffer handling. It aims to provide a unified NPU acceleration experience across major chipset providers and best-in-class GPU performance through advanced buffer interoperability for zero-copy operations. The framework also prioritizes superior generative AI inference, simplifying integration and boosting performance for large models.

Quick Start & Requirements

• Installation: Build from source using provided Docker scripts (docker_build/build_with_docker.sh).
• Prerequisites: Docker daemon. Cross-compilation for Android is supported within the Docker environment.
• Documentation: Get Started guide available.
• Early Access: LiteRT NPU Early Access Program (EAP) registration at g.co/ai/LiteRT-NPU-EAP.

Highlighted Details

• Supports a wide range of platforms including Android, iOS, Linux, macOS, Windows, and embedded systems, with varying levels of CPU, GPU, and NPU acceleration.
• Offers specialized solutions for Generative AI (LiteRT LM) and efficient model conversion from PyTorch (AI Edge Torch Converter, Torch Generative API).
• LiteRT V2 is currently in Alpha, with a roadmap targeting Beta by December 2025 and General Availability by Google I/O, May 2026.

Maintenance & Community

• Actively under development with a planned release cadence of minor releases every 4-6 weeks.
• Community support is available via GitHub Issues for bugs/features and GitHub Discussions for general questions. Contributions are welcomed via CONTRIBUTING.md.

Licensing & Compatibility

• Licensed under the Apache-2.0 License.
• Compatible with commercial use and closed-source linking.

Limitations & Caveats

LiteRT V2 is an alpha release, indicating potential instability and ongoing changes. Some hardware acceleration features (e.g., WebGPU, specific NPU support) are marked as "Coming soon."