Adlik  by Adlik

Deep learning inference acceleration framework

Created 6 years ago
806 stars

Top 43.9% on SourcePulse

GitHubView on GitHub
Project Summary

Adlik is an end-to-end framework designed to accelerate deep learning inference across cloud and embedded environments. It targets engineers and researchers seeking flexible, high-performance deployment of models developed in popular frameworks like TensorFlow, Keras, and PyTorch. Adlik streamlines the optimization and deployment pipeline, enabling efficient inference on diverse hardware platforms.

How It Works

Adlik operates through three core components: a Model Optimizer (featuring pruners and quantizers), a Model Compiler that applies techniques like pruning, quantization, and structural compression, and a Serving Engine that provides optimized runtimes tailored to specific deployment environments. This pipeline allows users to optimize models from various sources (H5, CheckPoint, FrozenGraph, ONNX, SavedModel) into formats like OpenVino, TensorFlow Lite, and TensorRT, which are then served by Adlik's engines for accelerated inference.

Quick Start & Requirements

The quickest way to get started is by using pre-built Docker images available on Alibaba Cloud for both model compilation and serving. Users can pull these images using docker pull registry.cn-beijing.aliyuncs.com/adlik/model-compiler:v1.0 or similar serving image names. Building from source requires Git, Bazel, and Python 3.x, with specific CUDA, cuDNN, and TensorRT versions needed for GPU-enabled builds, and OpenVINO for OpenVINO runtime builds.

Highlighted Details

  • Supports model optimization and compilation from TensorFlow, Keras, PyTorch, and PaddlePaddle.
  • Enables compilation to OpenVino, TensorFlow Lite, and TensorRT formats.
  • Offers serving engines for CPU and GPU, supporting runtimes including TFLite, TensorFlow, OpenVino, TensorRT, and LibTorch.
  • The framework demonstrates performance improvements on models like MNIST, ResNet50, InceptionV3, YoloV3, and Bert.

Maintenance & Community

No specific details regarding maintenance, community channels (like Discord/Slack), or notable contributors were found in the provided text.

Licensing & Compatibility

Adlik is licensed under the Apache License 2.0, which generally permits commercial use and integration into closed-source projects.

Limitations & Caveats

Building Adlik from source involves complex dependency management, particularly for GPU support requiring specific CUDA and TensorRT versions. Additionally, PaddlePaddle models are not supported by the TensorFlow or PyTorch serving engines.

Health Check
Last Commit

1 year ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
0 stars in the last 30 days

Explore Similar Projects

Starred by Shengjia Zhao Shengjia Zhao(Chief Scientist at Meta Superintelligence Lab), Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), and
14 more.

BIG-bench by google

0.1%
3k
Collaborative benchmark for probing and extrapolating LLM capabilities
Created 4 years ago
Updated 1 year ago
Starred by Aravind Srinivas Aravind Srinivas(Cofounder of Perplexity), Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), and
16 more.

text-to-text-transfer-transformer by google-research

0.1%
6k
Unified text-to-text transformer for NLP research
Created 6 years ago
Updated 5 months ago
Feedback? Help us improve.