This C++ toolkit provides a lightweight, unified interface for over 100 pre-trained AI models, including object detection, segmentation, face analysis, and style transfer. It targets C++ developers seeking to integrate diverse AI capabilities into their applications with minimal dependencies and a consistent API. The primary benefit is rapid prototyping and deployment of complex AI features using a single, easy-to-use library.

How It Works

The toolkit acts as a high-level abstraction layer over popular inference engines like ONNX Runtime, MNN, and TensorRT. It offers a C++ API with a lite::cv::Type::Class syntax for accessing various models. This approach simplifies model integration by abstracting away the complexities of individual engine APIs and providing a consistent interface for model loading, inference, and post-processing.

Quick Start & Requirements

• Install: Download prebuilt libraries from releases or build from source using sh ./build.sh.
• Prerequisites: OpenCV is a default dependency. TensorRT support requires NVIDIA GPU, CUDA 12.x, and TensorRT 10.x+.
• Setup: Prebuilt libraries and test resources are available via release tags. Building from source is supported on Linux.
• Docs: Examples, Model Zoo

Highlighted Details

• Supports 300+ C++ implementations and 500+ weights across ONNX, MNN, NCNN, and TNN formats.
• Offers TensorRT integration for NVIDIA GPU acceleration.
• Allows mixing toolkit models with custom ONNX Runtime or MNN implementations.
• Extensive model coverage includes YOLO variants, face recognition (ArcFace, CosFace), matting, and more.

Maintenance & Community

The project is primarily maintained by @wangzijian1010. The README mentions focus on LLM/VLM Inference and links to related projects like "Awesome-LLM-Inference" and "LeetCUDA".

Licensing & Compatibility

The project is licensed under the GNU General Public License v3.0 (GPL-3.0). This license is copyleft, meaning derivative works must also be open-sourced under the same license, potentially restricting commercial use in closed-source applications.

Limitations & Caveats

The project's primary dependency, OpenCV, can be substantial. While prebuilt libraries are available, building from source may require significant time and system configuration. The GPL-3.0 license imposes strict copyleft requirements that may not be suitable for all commercial applications.