OneDiff is an acceleration library designed to speed up diffusion models for users of popular UIs like ComfyUI and Hugging Face Diffusers. It offers PyTorch code compilation and optimized GPU kernels, aiming to provide significant performance gains with minimal code changes.

How It Works

OneDiff leverages PyTorch module compilation, specifically through its OneFlow backend or the optional Nexfort compiler. This process compiles PyTorch code into optimized kernels, reducing overhead from dynamic Python execution and enabling faster inference. The compilation can be done offline and the results loaded for online serving, supporting dynamic input shapes without recompilation penalties.

Quick Start & Requirements

• Installation: python3 -m pip install --pre onediff (or from source for plugins).
• Prerequisites: PyTorch, Hugging Face Diffusers, and a compiler backend (OneFlow or Nexfort). Requires NVIDIA GPUs (3090 RTX/4090 RTX/A100/A800/A10 etc.). CUDA 11.8, 12.1, or 12.2 are supported for OneFlow.
• Setup: Installation involves installing PyTorch, diffusers, a compiler backend, and OneDiff itself.
• Documentation: Documentation

Highlighted Details

• Up to 1.7x speedup reported for Kolors, DiT, SD3, PixArt, and Latte models.
• Supports acceleration for SD 1.5-XL, SDXL Turbo, LCM, LoRA, ControlNet, SVD, and InstantID.
• Integrates with ComfyUI, Hugging Face Diffusers, and Stable Diffusion web UI.
• Offers features like dynamic image size support and fast LoRA switching.

Maintenance & Community

• Active development with recent updates for DiT and Kolors acceleration.
• Community support via Discord and GitHub Issues.
• Discord

Licensing & Compatibility

• The repository does not explicitly state a license in the README.

Limitations & Caveats

• Windows support is limited to WSL. Compatibility with Ascend GPUs is in progress. Some features like SDXL DeepCache are in alpha status.