This repository provides training-free methods, LoRA Switch and LoRA Composite, for integrating multiple Low-Rank Adaptation (LoRA) elements into image generation. It targets users of diffusion models, particularly those working with Stable Diffusion, enabling precise control over character, style, and object composition in generated images.

How It Works

The project introduces two novel techniques, LoRA Switch and LoRA Composite, as alternatives to traditional LoRA merging. LoRA Switch sequentially applies LoRAs across diffusion steps, while LoRA Composite offers a more integrated approach. These methods aim to improve the accuracy and flexibility of combining multiple LoRAs without requiring additional training.

Quick Start & Requirements

• Installation: Use conda create --name multi-lora python=3.10, conda activate multi-lora, and pip install -r requirements.txt.
• Pre-trained LoRAs: Download ComposLoRA.zip and place it in the models folder.
• Dependencies: Requires Python 3.10, CUDA, and PyTorch. The examples utilize Hugging Face's diffusers library.
• Resources: A GPU is necessary for image generation. The ComposLoRA.zip file contains 22 pre-trained LoRAs.
• Documentation: example.py, lcm_example.py, lcm_lora_example.py, sdxl_example.py, compose_lora.py, evaluate.py.

Highlighted Details

• Supports composition of 2-5 LoRAs.
• Compatible with SD1.5, SDXL, LCM, and LCM-LoRA models.
• Includes a testbed with 22 pre-trained LoRAs covering characters, clothing, styles, backgrounds, and objects.
• Offers comparative evaluation scripts using GPT-4V and human evaluation methods.

Maintenance & Community

The project is associated with authors from Microsoft. Links to community channels are not explicitly provided in the README.

Licensing & Compatibility

The repository does not explicitly state a license. The code is presented for research purposes.

Limitations & Caveats

The README mentions a "position bias of GPT-4V" for evaluation, suggesting potential limitations in automated quality assessment. The project is presented as a research artifact, and commercial use implications are not detailed.