This repository provides a comprehensive zoo of pre-trained neural network models optimized for deployment on Rockchip's NPUs. It targets embedded systems developers and researchers needing to accelerate AI workloads on Rockchip hardware, offering a wide range of models for tasks like object detection, image classification, and speech recognition, with Python and C++ inference examples.

How It Works

The zoo leverages the RKNN-Toolkit2 for model conversion, enabling the deployment of various deep learning architectures (e.g., YOLO, ResNet, Whisper) onto Rockchip's NPU platforms. Models are provided in RKNN format, supporting FP16 and INT8 quantization for performance and efficiency gains. The toolkit facilitates model export and inference via Python and C++ APIs, abstracting hardware-specific complexities.

Quick Start & Requirements

• Installation: Requires cloning the repository and installing the RKNN-Toolkit2.
• Prerequisites:
  • RKNN-Toolkit2 (see https://github.com/airockchip/rknn-toolkit2).
  • Android NDK (r18/r19 recommended) for Android demos.
  • Specific Linux GCC toolchains (e.g., gcc-linaro-6.3.1 for aarch64) for Linux demos.
  • Pre-trained model files available via a provided download link.
• Setup: Compilation environment setup is critical and detailed in Compilation_Environment_Setup_Guide.md.

Highlighted Details

• Supports a broad range of Rockchip platforms including RK3562, RK3566, RK3568, RK3576, RK3588, RV1126B, RV1103, RV1106, RV1109, RV1126, and RK1808.
• Offers a diverse model catalog covering classification, object detection (YOLO variants), segmentation, OCR, speech recognition (Whisper, Wav2Vec2), and more.
• Provides detailed performance benchmarks (FPS) across different Rockchip NPU models for various quantized models.
• Includes build scripts (build-linux.sh, build-android.sh) for compiling inference demos tailored to specific targets and architectures.

Maintenance & Community

• Regular updates with new models and platform support (e.g., recent additions include YOLO11, Zipformer, MMS-TTS).
• Dependencies tied to specific RKNPU SDK versions (>=2.3.2 for latest).

Licensing & Compatibility

• Licensed under the Apache License 2.0, permitting commercial use and integration with closed-source applications.

Limitations & Caveats

• Compilation requires specific, potentially outdated, toolchains which may be challenging to set up.
• Performance benchmarks are based on maximum NPU frequency and may not include pre/post-processing times.
• Support for certain platforms (e.g., RV1103/RV1106) is noted as "limited."