This repository provides official sample applications and code examples for Google's LiteRT (formerly TensorFlow Lite) on-device machine learning framework. It targets engineers and developers seeking to implement ML and GenAI models efficiently on edge devices. The samples demonstrate two distinct API paradigms, enabling users to leverage either broad CPU compatibility or advanced hardware acceleration for superior performance.

How It Works

The project offers two primary API approaches: the interpreter_api/ for standard CPU-based execution across a wide range of platforms, and the compiled_model_api/ designed for advanced GPU/NPU acceleration. The CompiledModel API focuses on hardware acceleration, asynchronous execution, and efficient buffer management to deliver superior ML and GenAI performance, supporting both Ahead-of-Time (AOT) and Just-in-Time (JIT) compilation. The Interpreter API ensures broad compatibility with .tflite models across Android and iOS versions, utilizing the legacy Task Library.

Quick Start & Requirements

• Prerequisites: Android Studio (latest stable), Xcode (latest), Python 3.9+ with pip install ai-edge-litert.
• Running Compiled Model API Samples: Navigate to compiled_model_api/. Requires a device with a supported NPU (e.g., modern Pixel, Samsung, MediaTek/Qualcomm chips). Follow specific sub-folder instructions to enable hardware delegates.
• Running Interpreter API Samples: Navigate to interpreter_api/, open in Android Studio or Xcode, and build/run on a device.
• Documentation: LiteRT Overview: https://ai.google.dev/edge/litert, CompiledModel API Guide, Model Conversion guides are available.

Highlighted Details

• Demonstrates advanced GPU/NPU acceleration via the compiled_model_api for superior ML & GenAI performance.
• Provides distinct API paradigms: compiled_model_api for hardware acceleration and interpreter_api for broad CPU compatibility.
• Features sample applications including Image Classification, Object Detection, Segmentation, Audio Classification, and Digit Recognition.
• Supports platforms such as Android (Kotlin/C++), iOS (Swift/Objective-C), and Python (Raspberry Pi/Linux).

Maintenance & Community

Contributions are welcomed via pull requests as outlined in CONTRIBUTING.md. No specific community channels (e.g., Discord, Slack) are listed in the README.

Licensing & Compatibility

The project is licensed under the Apache License 2.0, which is generally permissive for commercial use and integration into closed-source projects.

Limitations & Caveats

This is a sample repository provided "as is" without warranty. For Generative AI and Large Language Models (LLMs), users are directed to the separate LiteRT-LM repository. The compiled_model_api requires specific NPU hardware, limiting its immediate applicability on devices lacking such accelerators.