This repository provides a PyTorch implementation of SVDiff, a method for compact parameter space fine-tuning of diffusion models. It enables efficient single-subject generation and single-image editing with significantly reduced model sizes and faster training compared to LoRA.

How It Works

SVDiff fine-tunes diffusion models by learning low-rank spectral shifts in the parameter space, specifically targeting the U-Net and text encoder. This approach allows for a more compact representation of learned concepts, resulting in smaller checkpoint files (1.2MB vs. 3.1MB for LoRA) and fewer trainable parameters. The method is designed to achieve comparable or better results with fewer training steps.

Quick Start & Requirements

• Install via pip: pip install svdiff-pytorch
• Alternatively, clone the repo and install requirements: git clone https://github.com/mkshing/svdiff-pytorch && pip install -r requirements.txt
• Requires PyTorch and Hugging Face diffusers.
• GPU with CUDA is recommended for training and inference.
• Official documentation and a Gradio UI demo are available.

Highlighted Details

• Achieves 0.5M fewer trainable parameters and a 1.2MB file size compared to LoRA (3.1MB).
• Supports single-subject generation and single-image editing without DDIM inversion.
• Offers optional integration with ToMe for faster prior generation during training.
• Includes a Gradio UI for both training and inference.

Maintenance & Community

• The project was last updated in April 2023.
• Key dependencies include Hugging Face diffusers.
• Links to the original paper and related works (LoRA, ToMe) are provided.

Licensing & Compatibility

• The repository does not explicitly state a license in the README.
• Compatibility with commercial or closed-source projects is not specified.

Limitations & Caveats

• The project appears to be a snapshot from April 2023, and future maintenance or updates are not guaranteed.
• Features like "Support multiple spectral shifts" and "SVDiff + LoRA" are marked as TODO.