bolt  by huawei-noah

Deep learning library for high-performance, heterogeneous deployment

created 5 years ago
954 stars

Top 39.3% on sourcepulse

GitHubView on GitHub
Project Summary

Bolt is a high-performance, lightweight deep learning inference library designed for efficient deployment across a wide range of hardware and model formats. It targets developers and researchers needing to optimize neural network performance for edge devices and servers, offering significant speedups and reduced resource consumption.

How It Works

Bolt employs a graph optimization engine and efficient thread affinity settings to maximize inference speed. It supports various numerical precisions (FP32, FP16, INT8, BNN) and model formats (Caffe, ONNX, TFLite, TensorFlow), enabling broad compatibility. Its architecture is built for heterogeneous flexibility, allowing it to leverage specific hardware acceleration features.

Quick Start & Requirements

Build and installation is performed via the install.sh script, with various target platforms and precision options. For example, ./install.sh --target=android-aarch64 for Android ARMv8. Detailed instructions for building with specific compilers and deploying models are available in the docs directory.

Highlighted Details

  • Claims 15%+ performance improvement over existing open-source acceleration libraries.
  • Supports a wide array of platforms including ARM (v7, v8, v8.2+, v9), x86 (AVX2, AVX512), and various GPUs (Mali, Qualcomm, Intel, AMD).
  • Offers support for NLP tasks, including BERT and TTS, in addition to common CV applications.
  • Includes an on-device training module (beta) for select models like LeNet, MobileNet_v1, and ResNet18.

Maintenance & Community

The project is developed by Huawei Noah's Ark Lab. Community support is available via QQ group: 833345709.

Licensing & Compatibility

Licensed under the MIT License, permitting commercial use and integration with closed-source projects.

Limitations & Caveats

The on-device training feature is currently in beta and supports a limited set of models. The default static library linking may cause issues on some platforms, with a --shared option available for shared library linking.

Health Check
Last commit

3 months ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
13 stars in the last 90 days

Explore Similar Projects

Starred by George Hotz George Hotz(Author of tinygrad; Founder of the tiny corp, comma.ai), Anton Bukov Anton Bukov(Cofounder of 1inch Network), and
16 more.

tinygrad by tinygrad

0.1%
30k
Minimalist deep learning framework for education and exploration
created 4 years ago
updated 21 hours ago
Starred by Aravind Srinivas Aravind Srinivas(Cofounder of Perplexity), Stas Bekman Stas Bekman(Author of Machine Learning Engineering Open Book; Research Engineer at Snowflake), and
12 more.

DeepSpeed by deepspeedai

0.2%
40k
Deep learning optimization library for distributed training and inference
created 5 years ago
updated 1 day ago
Feedback? Help us improve.