X-Adapter addresses the challenge of adapting existing diffusion model plugins (like ControlNet or LoRA) to newer, upgraded base models without costly retraining. It offers universal compatibility, allowing users to leverage their existing plugin ecosystem with enhanced diffusion models, saving significant time and computational resources.

How It Works

X-Adapter employs a novel adapter mechanism that bridges the architectural differences between older and newer diffusion model versions. By introducing lightweight, trainable adapter modules, it effectively "translates" the conditioning signals from plugins trained on older models to be understood by newer base models. This approach avoids full model retraining, making plugin adaptation efficient and accessible.

Quick Start & Requirements

• Install: conda create -n xadapter python=3.10, conda activate xadapter, pip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 --extra-index-url https://download.pytorch.org/whl/cu116, pip install -r requirements.txt.
• Prerequisites: CUDA 11.6, PyTorch 1.13.1, Python 3.10. xformers is highly recommended for efficiency.
• Resources: Requires downloading pre-trained models (Stable Diffusion, ControlNet, LoRA) and placing them in ./checkpoint/X-Adapter.
• Docs: Tutorial

Highlighted Details

• Enables plugins pretrained on older diffusion models (e.g., SD1.5) to work with upgraded models (e.g., SDXL).
• Supports various plugin types including ControlNet, LoRA, and ControlNet-Tile.
• Inference scripts provided for different plugin types.
• Official implementation of the CVPR 2024 paper "X-Adapter: Adding Universal Compatibility of Plugins for Upgraded Diffusion Model".

Maintenance & Community

• Official implementation by the authors of the CVPR 2024 paper.
• Links to Hugging Face for checkpoint models.

Licensing & Compatibility

• The repository itself does not explicitly state a license in the provided README. The underlying models (Stable Diffusion, ControlNet, LoRA) have their own licenses.

Limitations & Caveats

The installation requires a specific older version of PyTorch (1.13.1) with CUDA 11.6, which may conflict with other CUDA-dependent projects. The README does not detail compatibility with newer PyTorch versions or CUDA toolkits.