HCP-Diffusion is a comprehensive toolbox for diffusion models, targeting researchers and power users who need a flexible and extensible framework for training and experimentation. It simplifies complex workflows by allowing users to define and combine various training techniques, such as LoRA, DreamBooth, and ControlNet, within a single Python configuration file.

How It Works

The framework utilizes the RainbowNeko Engine, which processes Python-based configuration files. This approach allows for direct function and class calls within configurations, enabling inheritance and dynamic instantiation of components. This design promotes extensibility and user-friendliness, simplifying the management of diverse training methodologies and model architectures.

Quick Start & Requirements

• Install via pip: pip install hcpdiff
• Initialize configuration: hcpinit
• Install from source: git clone https://github.com/7eu7d7/HCP-Diffusion.git && cd HCP-Diffusion && pip install -e .
• Optional: Install xformers for memory reduction and acceleration.
• Official documentation: 📘English document

Highlighted Details

• Supports Stable Diffusion 1.5, SDXL, and PixArt, with FLUX and SD3 in development.
• Offers extensive fine-tuning capabilities including layer-wise LoRA configuration, multi-token prompt-tuning, and custom optimizers/LR schedulers.
• Integrates with Hugging Face Accelerate, Colossal-AI, and xFormers for training acceleration.
• Implements DreamArtist++ for controllable one-shot text-to-image generation with a single image.
• Supports various dataset features like Aspect Ratio Bucketing and multi-source datasets.

Maintenance & Community

Maintained by HCP-Lab at Sun Yat-sen University.

Licensing & Compatibility

The repository does not explicitly state a license in the README. Users should verify licensing for commercial use or closed-source integration.

Limitations & Caveats

Automatic evaluation metrics like FID and CLIP Score are still in development. Support for webdataset is also in development.