openvino2tensorflow  by PINTO0309

Model conversion tool for AI inference

created 4 years ago
342 stars

Top 81.9% on sourcepulse

GitHubView on GitHub
Project Summary

This repository provides a comprehensive tool for converting deep learning models between various formats, primarily focusing on bridging ONNX/OpenVINO IR to TensorFlow and its ecosystem (SavedModel, TFLite, TFJS, etc.). It aims to simplify the complex model conversion process for users, especially those encountering difficulties with standard ONNX-to-TensorFlow tools, and specifically addresses issues with Transpose operations.

How It Works

The core functionality revolves around a Python script that orchestrates a multi-step conversion pipeline. It leverages specialized TensorFlow and OpenVINO libraries, along with tools like TensorRT, CoreML, and EdgeTPU compilers. The process typically involves converting from PyTorch (NCHW) to ONNX (NCHW), then to OpenVINO IR (NCHW), and finally to TensorFlow formats (NHWC/NCHW), with extensive support for intermediate and final format conversions. A key advantage is its ability to handle complex layer transformations and shape manipulations, offering workarounds for common issues like Transpose and Reshape operations through configuration files.

Quick Start & Requirements

  • Primary install / run command: Docker is strongly recommended.
    • docker pull ghcr.io/pinto0309/openvino2tensorflow:latest
    • docker run -it --rm -v $(pwd):/home/user/workdir ghcr.io/pinto0309/openvino2tensorflow:latest
  • Non-default prerequisites: Python 3.8+, TensorFlow v2.10.0+, PyTorch v1.12.1+, OpenVINO 2022.1.0, TensorRT 8.4.0+. NVIDIA GPU (CUDA) and Intel iHD GPU (OpenCL) are supported. Docker installation is required for the recommended setup.
  • Estimated setup time: Docker image pull can take time depending on network speed. Building from source or setting up the host environment requires installing numerous dependencies.
  • Links: Official Quick Start, Supported Layers, Usage Examples

Highlighted Details

  • Supports conversion to SavedModel, TFLite (multiple quantization types), TFJS, TensorRT, CoreML, EdgeTPU, ONNX, and Protocol Buffer (.pb).
  • Offers advanced features like weight replacement via JSON configuration and layer-specific output debugging.
  • Includes specific optimizations and workarounds for common issues like Transpose, Reshape, and Swish/HardSwish operations.
  • Provides Dockerfiles for various hardware acceleration setups (NVIDIA GPU, Intel iGPU) and GUI/camera access.

Maintenance & Community

  • The project is actively maintained by PINTO0309.
  • Links to community resources are not explicitly provided in the README.

Licensing & Compatibility

  • The README does not explicitly state a license. Users should verify licensing for commercial use.

Limitations & Caveats

  • The project is described as "a tool in the making, so there are lots of bugs."
  • Specific operations like 2D/3D/5D Tensor Reshape and Transpose have known issues, requiring manual configuration via JSON for workarounds.
  • Some operations like Conv3D, Unsqueeze, Range, NonMaxSuppression, and GatherElements are marked as Work In Progress (WIP) or have limited support.
Health Check
Last commit

2 years ago

Responsiveness

1 day

Pull Requests (30d)
0
Issues (30d)
0
Star History
2 stars in the last 90 days

Explore Similar Projects

Starred by Nat Friedman Nat Friedman(Former CEO of GitHub), Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and
6 more.

FasterTransformer by NVIDIA

0.2%
6k
Optimized transformer library for inference
created 4 years ago
updated 1 year ago
Starred by Patrick von Platen Patrick von Platen(Core Contributor to Hugging Face Transformers and Diffusers), Michael Han Michael Han(Cofounder of Unsloth), and
1 more.

ktransformers by kvcache-ai

0.4%
15k
Framework for LLM inference optimization experimentation
created 1 year ago
updated 2 days ago
Starred by Andrej Karpathy Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n), Nat Friedman Nat Friedman(Former CEO of GitHub), and
32 more.

llama.cpp by ggml-org

0.4%
84k
C/C++ library for local LLM inference
created 2 years ago
updated 16 hours ago
Feedback? Help us improve.