This Rust implementation of the Hugging Face Diffusers API provides a native alternative for running Stable Diffusion models. It targets developers and researchers seeking to integrate diffusion models into Rust applications or explore their capabilities without Python dependencies. The library leverages the tch crate for PyTorch bindings, enabling efficient execution on both CPU and GPU.

How It Works

The project directly implements the core components of the Stable Diffusion pipeline, including the U-Net, VAE, and CLIP text encoder, using Rust and tch. It supports various schedulers like DDIM and allows for image-to-image generation and inpainting by loading specific model weights and masks. The architecture is designed for flexibility, enabling users to specify which components run on the CPU or GPU to manage memory constraints.

Quick Start & Requirements

• Install via cargo install.
• Requires Rust toolchain, libtorch (PyTorch C++ API), and model weights downloaded from Hugging Face.
• Example run command: cargo run --example stable-diffusion --features clap -- --prompt "A rusty robot holding a fire torch."
• Official Python script available for weight download: python3 ./scripts/get_weights.py

Highlighted Details

• Supports Stable Diffusion v1.5 and v2.1.
• Implements Image-to-Image and Inpainting pipelines.
• Includes ControlNet integration for guided image generation.
• Offers CPU/GPU configuration for memory management (e.g., PYTORCH_CUDA_ALLOC_CONF).

Maintenance & Community

• Developed by LaurentMazare.
• No explicit community links (Discord/Slack) or roadmap mentioned in the README.

Licensing & Compatibility

• The README does not explicitly state a license. The tch crate is typically MIT licensed, but the project's specific licensing requires verification.
• Compatibility for commercial use is dependent on the underlying tch crate license and any potential restrictions on model weights.

Limitations & Caveats

The project requires manual downloading and placement of model weights. Memory optimization for GPUs with less than 8GB VRAM is complex, involving fp16 weights and specific component offloading. ControlNet requires additional specific weights.