FashionChameleon provides a real-time and interactive framework for human-garment customization in streaming autoregressive video generation. It targets researchers and developers in video synthesis and fashion technology, enabling rapid, on-the-fly changes to garments within videos with significant speedups over prior art.

How It Works

The framework builds upon the Wan2.2-TI2V-5B model, incorporating three key components: a teacher model leveraging in-context learning, streaming distillation also with in-context learning, and a novel training-free KV-cache rescheduling mechanism for interactive inference. To address frame quality degradation during generation, it employs gradient-reweighted DMD, dynamically adjusting gradient supervision based on frame quality to enhance extrapolation consistency.

Quick Start & Requirements

• Installation: Clone the repository, create and activate a Conda environment (python=3.10), and install dependencies via pip install -r requirements.txt.
• Backbone Weights: Download Wan2.2-TI2V-5B from Hugging Face, clip-vit-large-patch14 weights, and the improved-aesthetic-predictor weights.
• Hardware: Requires GPUs (demonstrated on H200) and CUDA. Training commands specify multi-GPU usage (e.g., 4 or 8 GPUs).
• Links: Project Page and Technical Report are available (though URLs are not directly provided in the README snippet).

Highlighted Details

• Achieves real-time generation at 23.8 FPS on a single H200 GPU, reportedly 30–180× faster than existing methods.
• Enables interactive multi-garment customization without requiring video data containing multiple garments.
• Introduces gradient-reweighted DMD to improve extrapolation consistency by prioritizing supervision for lower-quality frames.

Maintenance & Community

The code has been released, along with a technical report and HGC-Bench dataset. A "Todo" list indicates pending releases for checkpoints and further interactive inference enhancements. No explicit community channels (like Discord/Slack) or active contributor information are detailed in the provided text.

Licensing & Compatibility

The repository README does not specify a software license. This omission requires clarification for any adoption decision, particularly concerning commercial use or integration into closed-source projects.

Limitations & Caveats

The project is actively under development, with several items listed in the "Todo" section, including the release of checkpoints and further refinement of interactive inference techniques. The core framework relies on specific pre-trained backbones, and compatibility with other models is not detailed.