This project provides a C++ toolbox, AiDB, for deploying deep learning models across various inference frameworks like ONNX Runtime, MNN, NCNN, TNN, PaddleLite, and OpenVINO. It offers a unified interface for multiple mainstream deep learning models, including YoloX/V7/V8, OCR, MobileViT, and Stable Diffusion, targeting developers needing efficient cross-platform model deployment.

How It Works

AiDB abstracts the complexities of different inference engines into a single, configurable interface. Users can select their desired backend (e.g., ONNX Runtime, MNN) via configuration files, simplifying the process of switching between or utilizing multiple frameworks. This approach aims to streamline the deployment workflow for a wide range of AI models across diverse hardware and operating systems.

Quick Start & Requirements

• Installation: Docker is recommended (docker pull mister5ive/ai.deploy.box). Alternatively, build from source using CMake with configurable options for backends and language APIs.
• Dependencies: OpenCV is a key dependency. Specific model backends may have additional requirements.
• Resources: Pre-trained models are available via MEGA and Baidu links.
• Demos: Multiple demos are provided for PC (Qt), Android (Kotlin), Lua, and Go, with Webassembly and Google Colab options available. See Demo Repo.

Highlighted Details

• Supports six major inference frameworks: ONNX Runtime, MNN, NCNN, TNN, PaddleLite, and OpenVINO.
• Offers unified interfaces for numerous models including Yolo series, SCRFD, MobileSAM, Stable Diffusion, and more.
• Provides deployment examples for C++, Python, Lua, and Go.
• Includes specific demos for face detection, landmark detection, alignment, parsing, OCR, object detection, and GANs.

Maintenance & Community

The project is maintained by @TalkUHulk. Contributions are welcomed via issues and pull requests.

Licensing & Compatibility

The project is licensed under the MIT license, permitting commercial use and integration with closed-source projects.

Limitations & Caveats

The README notes potential build issues on Android related to RTTI and specific backend configurations (MNN, Paddle-Lite, OpenVINO), with workarounds provided. iOS support is explicitly limited due to resource constraints.