This project provides a multi-threaded inference demo for YOLOv8 on the RK3588 platform, targeting developers and researchers working with embedded AI and computer vision on this specific hardware. It offers up to 100 FPS inference for video files and camera feeds using the Yolov8n model, aiming to showcase efficient object detection performance.

How It Works

The demo leverages multi-threading for parallel processing of inference tasks. It supports reading from video files and camera feeds, with specific executables for each. The core approach involves using the Yolov8n model, converted to ONNX format, and then potentially to RKNN for optimized execution on the RK3588's NPU.

Quick Start & Requirements

• Install: Clone the repository and build using CMake. A cross-compilation environment is required.
• Prerequisites: RK3588 platform, cross-compilation toolchain, rknn-toolkit2 (beta 2.0.0b12 recommended for attention operator support), libbytetrack. Runtime libraries can be installed via a custom APT repository.
• Model Files: Download links for .pt and ONNX models are provided.
• Links:
  • GitHub: https://github.com/kaylorchen/rk3588-yolo-demo
  • Bilibili (explanation): https://space.bilibili.com/327258623
  • Blog (model conversion): blog.kaylordut.com

Highlighted Details

• Yolov8n inference up to 100 FPS on RK3588.
• Supports video file, camera feed, and image file inference.
• Benchmarks provided for various YOLOv8/v10 models and rknn-toolkit versions.
• Docker image available for rknn-toolkit2 beta with attention operator support.

Maintenance & Community

• The author maintains a newer, more compatible repository for inference frameworks.
• QQ groups are available for community support (Group 1: 957577822, Group 2: 546943464).
• Contact email for project collaboration: kaylor.chen@qq.com.

Licensing & Compatibility

• The repository itself does not explicitly state a license in the README.
• Model conversion instructions are provided for ONNX and RKNN formats, suggesting compatibility with Rockchip's RKNN ecosystem.

Limitations & Caveats

The README notes that the code in this specific repository is not optimal and may have ordering issues, recommending a newer repository for a more robust solution. Older versions of rknn-toolkit2 may run attention steps on the CPU, impacting performance.