Summary

The RDK Model Zoo provides deployment routines and Python APIs for mainstream AI algorithms on D-Robotics RDK platforms (X5, S100, X3). It simplifies the process of exporting D-Robotics .bin models and enables inference, offering end-to-end solutions that cover data collection, training, conversion, and deployment for various model types including image classification, object detection, instance segmentation, and LLMs.

How It Works

Developed based on the RDK framework, the zoo facilitates exporting models into D-Robotics' proprietary .bin format. It offers Python APIs for efficient inference on these models, leveraging the platform's hardware accelerators. Some routines encompass the complete AI lifecycle, from data preparation to final deployment, streamlining the development workflow for embedded AI applications on D-Robotics hardware.

Quick Start & Requirements

• Primary Install/Run: Clone the repository (git clone https://github.com/D-Robotics/rdk_model_zoo). Install JupyterLab (pip install jupyterlab) and run demos via jupyter lab --allow-root --ip <IP_ADDRESS>. Python API inference requires pip install bpu_infer_lib_x5 (or _x3) from http://sdk.d-robotics.cc:8080/simple/.
• Prerequisites: D-Robotics RDK board (X5, S100, X3), RDK OS >= 3.2.3 (for X5), internet access, SSH client.
• Links: Main Repo: https://github.com/D-Robotics/rdk_model_zoo. S100 Repo: https://github.com/d-Robotics/rdk_model_zoo_s.

Highlighted Details

• Supports D-Robotics RDK X5, S100, and X3 platforms.
• Provides deployment routines for Image Classification, Object Detection, Instance Segmentation, and LLMs.
• Includes end-to-end pipeline examples for select models.
• Offers both Python and C/C++ inference APIs.

Maintenance & Community

Feedback and issues are primarily handled through the D-Robotics Developer Community. No specific details on contributors, sponsorships, or roadmaps are provided.

Licensing & Compatibility

The license type is not specified in the README. Compatibility is limited to D-Robotics RDK hardware. Commercial use implications are undetermined without a license.

Limitations & Caveats

Users must checkout specific branches (e.g., rdk_x3) if not using RDK X5. Python API inference performance is noted as weaker than C/C++ APIs. Model precision (mAP) may differ from external benchmarks due to fixed vs. dynamic shapes and calculation variations. Quantization and input format conversions can introduce precision loss. Operators unsupported by the BPU fallback to CPU computation. The README does not specify the project's license.