This repository provides tools and instructions for running Stable Diffusion models using ONNX FP16 on DirectML, targeting AMD and Intel GPUs. It simplifies VRAM usage and increases inference speed by converting PyTorch models to the ONNX FP16 format, enabling broader hardware compatibility for AI image generation.

How It Works

The project leverages ONNX Runtime with the DirectML execution provider to achieve hardware acceleration. It includes a conversion script (conv_sd_to_onnx.py) that transforms Hugging Face diffusers models into ONNX format, specifically optimizing for FP16 precision. This approach reduces VRAM footprint and enhances performance, making Stable Diffusion accessible on a wider range of GPUs, including those from AMD and Intel. The conversion process supports various optimizations like attention slicing and alternative VAEs.

Quick Start & Requirements

• Install: Clone the repository, create and activate a virtual environment, and install dependencies via pip install -r requirements.txt.
• Prerequisites: Python 3.10 or 3.11, Git, Hugging Face account and token (huggingface-cli login).
• Setup: Follow the detailed command-line instructions for cloning, environment setup, model conversion, and running test scripts.
• Docs: https://github.com/Amblyopius/Stable-Diffusion-ONNX-FP16

Highlighted Details

• Supports conversion of .ckpt and .safetensors files with optional .yaml config.
• Includes experimental support for ControlNet and InstructPix2Pix models.
• Offers options for VRAM reduction, including loading the Text Encoder or VAE on CPU.
• Demonstrates integration with a fork of OnnxDiffusersUI for a graphical interface.

Maintenance & Community

• The project actively tracks PyTorch and ONNX Runtime releases.
• GitHub Discussions are enabled for community questions.
• Mentions an onnxdiffusers channel on Discord for support.

Licensing & Compatibility

• The repository itself is likely under a permissive license (e.g., MIT, Apache), but the underlying Stable Diffusion models and dependencies may have their own licenses. The README does not explicitly state the license for this repository.
• Converted models are designed to be compatible with other DirectML-based ONNX implementations.

Limitations & Caveats

• ControlNet conversion regression testing is ongoing.
• The Euler Ancestral scheduler may produce non-deterministic results with ONNX due to its noise addition mechanism.
• For maximum performance, especially on AMD, users are directed to explore the SHARK project.