This repository provides CatVTON-Flux, a state-of-the-art virtual try-on solution combining CATVTON with the Flux inpainting model for realistic clothing transfer. It targets researchers and developers in computer vision and fashion technology, offering improved garment detail and text rendering.

How It Works

CatVTON-Flux leverages a concatenation-based approach, inspired by In-Context LoRA, for prompt engineering. It utilizes the Flux fill inpainting model for enhanced realism and accuracy in clothing transfer. The system can also extract and reconstruct garment fronts from images of people wearing them (try-off functionality).

Quick Start & Requirements

• Install: pip install -r requirements.txt within a conda environment (Python 3.10 recommended).
• Prerequisites: VRAM >= 40GB is required; 80GB recommended for training. The model is trained on the VITON-HD dataset.
• Demo: Run python app.py for Gradio demo with LoRA weights.
• Docs: CATVTON-FLUX-TRY-ON, CAT-TRYOFF-FLUX

Highlighted Details

• Achieved SOTA FID: 5.59 on VITON-HD dataset.
• Released LoRA weights with FID: 6.07 on VITON-HD.
• Offers both try-on and try-off (garment reconstruction) capabilities.
• Training code for Flux inpainting full parameter fine-tuning is available.

Maintenance & Community

The project has seen recent updates, including new try-on and try-off models, training notes, and ComfyUI support. Links to Hugging Face spaces for demos are provided.

Licensing & Compatibility

The code is licensed under the MIT License. Model weights inherit licenses from Flux Fill and VITON-HD.

Limitations & Caveats

High VRAM requirements (>= 40GB) are noted as a potential barrier. Training requires significant resources (2xH100 80GB).