This tool provides a comprehensive suite for parsing, editing, and profiling ONNX models, targeting AI researchers and developers working with large language models, diffusion models, and computer vision applications. It aims to simplify model optimization, memory compression, and performance analysis.

How It Works

The tool operates by leveraging a compute graph and shape engine to perform static analysis and transformations on ONNX models. It supports operations like constant folding and operator fusion for optimization. For profiling, it relies on shape inference to accurately calculate MACs and memory usage, enabling detailed performance analysis and memory compression techniques for activations and weights.

Quick Start & Requirements

• Install via pip: pip install onnx-tool or pip install --upgrade git+https://github.com/ThanatosShinji/onnx-tool.git
• Requires Python >= 3.6.
• For installation issues, try pip install onnx==1.8.1 first.
• Official examples and usage guides are available in the benchmark/ and data/ directories.

Highlighted Details

• Supports a wide range of models including LLMs (GPT, LLaMa, Phi-3), diffusion models (Stable Diffusion), and various CV/Audio models.
• Achieves significant memory compression (e.g., 5% for activations, substantial weight compression via quantization).
• Offers rapid shape inference and MACs statistics, profiling 10 Hugging Face models in under a second.
• Enables editing of ONNX graph structure, operator attributes, and tensor data.

Maintenance & Community

No specific community channels (Discord/Slack) or notable contributors are listed in the README.

Licensing & Compatibility

The README does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The tool does not support ONNX Loop or Sequence types. Dynamic input shapes can affect MACs calculations, with specific input shapes for benchmark results provided in data/public/config.py.