This repository provides the official code for "Concept Sliders," enabling precise control over diffusion models through LoRA adaptors. It targets researchers and practitioners in generative AI who need to fine-tune model outputs for specific attributes or concepts, offering a method to disentangle and manipulate these elements with greater accuracy.

How It Works

Concept Sliders leverage LoRA (Low-Rank Adaptation) to inject controllable "sliders" into pre-trained diffusion models. The approach involves training small, specialized adapter modules that modify the model's behavior based on defined positive, negative, and neutral concepts. This allows for fine-grained adjustments to specific attributes (e.g., age, gender, object size) without retraining the entire diffusion model, leading to more efficient and targeted control.

Quick Start & Requirements

• Installation: Clone the repository and install dependencies using conda create -n sliders python=3.9, conda activate sliders, git clone https://github.com/rohitgandikota/sliders.git, cd sliders, and pip install -r requirements.txt.
• Prerequisites: Python 3.9+, Conda. Windows users should refer to specific setup guidelines.
• Demos: Colab Demo, Huggingface Demo, SliderSpace (automatic slider extraction).
• FLUX Support: Experimental support for FLUX-1 models requires pip install -r flux-sliders/flux-requirements.txt.

Highlighted Details

• Supports training of both textual and visual concept sliders for SD-1.x, SD-2.x, and SD-XL models.
• Offers GPT-4 integration for generating text prompts for sliders.
• Includes notebooks for evaluating trained models and editing real images using null inversion.
• Demonstrates integration with ControlNet.

Maintenance & Community

The project is associated with the ECCV 2024 paper "Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models." Updates include experimental FLUX support and the new SliderSpace feature for automatic slider extraction.

Licensing & Compatibility

The repository does not explicitly state a license in the provided README. Users should verify licensing for commercial use or integration into closed-source projects.

Limitations & Caveats

FLUX model support is experimental and may not perform as well as SDXL. The README does not detail specific hardware requirements beyond standard Python environments, but diffusion model training typically benefits from GPUs.