Deep learning library for high-performance, heterogeneous deployment
Top 39.3% on sourcepulse
Bolt is a high-performance, lightweight deep learning inference library designed for efficient deployment across a wide range of hardware and model formats. It targets developers and researchers needing to optimize neural network performance for edge devices and servers, offering significant speedups and reduced resource consumption.
How It Works
Bolt employs a graph optimization engine and efficient thread affinity settings to maximize inference speed. It supports various numerical precisions (FP32, FP16, INT8, BNN) and model formats (Caffe, ONNX, TFLite, TensorFlow), enabling broad compatibility. Its architecture is built for heterogeneous flexibility, allowing it to leverage specific hardware acceleration features.
Quick Start & Requirements
Build and installation is performed via the install.sh
script, with various target platforms and precision options. For example, ./install.sh --target=android-aarch64
for Android ARMv8. Detailed instructions for building with specific compilers and deploying models are available in the docs
directory.
Highlighted Details
Maintenance & Community
The project is developed by Huawei Noah's Ark Lab. Community support is available via QQ group: 833345709.
Licensing & Compatibility
Licensed under the MIT License, permitting commercial use and integration with closed-source projects.
Limitations & Caveats
The on-device training feature is currently in beta and supports a limited set of models. The default static library linking may cause issues on some platforms, with a --shared
option available for shared library linking.
3 months ago
Inactive