HairCLIP provides a PyTorch implementation for editing hairstyle and hair color using text descriptions or reference images. It targets researchers and developers in computer vision and generative AI, enabling flexible and unified hair manipulation within a single framework.

How It Works

HairCLIP leverages a StyleGAN architecture, specifically StyleCLIP, to manipulate latent codes. It incorporates CLIP for text-based guidance and an IR-SE50 model for identity preservation. The approach allows for individual or joint editing of hairstyle and color, offering a novel way to control specific attributes via diverse input modalities.

Quick Start & Requirements

• Install: conda install --yes -c pytorch pytorch=1.7.1 torchvision cudatoolkit=11.0, pip install ftfy regex tqdm, pip install git+https://github.com/openai/CLIP.git, pip install tensorflow-io.
• Prerequisites: PyTorch 1.7.1, CUDA 11.0, Python (version not specified but implied by PyTorch/CUDA), pre-trained models (HairCLIP, StyleGAN, IR-SE50), and latent codes (CelebA-HQ).
• Setup: Requires downloading multiple pre-trained models and datasets.
• Demo: https://replicate.com/r/wtybest/hairclip

Highlighted Details

• Supports individual or joint editing of hairstyle and hair color.
• Conditional inputs can be text descriptions or reference images.
• Based on the CVPR 2022 paper "HairCLIP: Design Your Hair by Text and Reference Image".
• A newer version, HairCLIPv2 (ICCV 2023), is also mentioned for improved performance and interaction modalities.

Maintenance & Community

• The project is associated with authors from the University of Science and Technology of China and Microsoft Cloud AI.
• Mentions a CVPR 2022 acceptance and an ICCV 2023 acceptance for HairCLIPv2.
• No explicit community links (Discord, Slack) are provided in the README.

Licensing & Compatibility

• The repository is released under an unspecified license. The code is based on StyleCLIP, which has its own licensing.
• Commercial use is not explicitly addressed.

Limitations & Caveats

• The provided implementation strictly supports batch size and test batch size of 1.
• Training requires significant setup with multiple auxiliary models and latent code inversions.
• The README mentions a potential need to pre-train a text-based hair color HairCLIP for certain augmentation strategies.