FaceChain is a deep-learning toolchain for generating personalized digital human portraits, targeting users who want to create custom avatars with high fidelity and stylistic flexibility. It offers a fast, train-free approach to identity-preserved portrait generation, compatible with popular tools like ControlNet and LoRAs.

How It Works

FaceChain utilizes a novel "train-free" pipeline, specifically the Face Adapter with Decoupled Training (FACT) method. Unlike traditional methods requiring extensive training data for each identity, FACT uses a single input photo and a parameter-efficient adapter module. This adapter, integrated into the Stable Diffusion U-Net via attention mechanisms, injects identity information alongside text prompts. The decoupled training strategy separates face information from the image and identity from the face, using a Transformer-based encoder (TransFace) and a novel FAIR loss function to improve image quality, text adherence, and controllability.

Quick Start & Requirements

• Installation: Recommended via ModelScope notebook (pip install modelscope) or Docker. Integration with stable-diffusion-webui is also supported via extensions.
• Prerequisites: Python 3.8/3.10, PyTorch 2.0.0+, CUDA 11.7, CUDNN 8+. Nvidia GPU (A10 24G verified). Jemalloc is recommended for memory optimization.
• Resources: A Colab notebook is available for quick experimentation.
• Docs: ModelScope Notebook, Docker Guide, Colab Demo.

Highlighted Details

• FACT offers 10-second portrait generation from a single photo.
• Seamless integration with ControlNet and LoRAs for enhanced customization.
• Supports multiple inference pipelines: Python scripts, Gradio interface, and sd webui.
• Achieved CVPR 2024 acceptance for related works (ImagineID, SuDe) and NeurIPS 2024 for TopoFR.

Maintenance & Community

The project is actively developed by ModelScope, with significant contributions and recognition, including Alibaba's Outstanding Open Source Project awards. Community support channels are not explicitly listed in the README.

Licensing & Compatibility

Licensed under the Apache License (Version 2.0), permitting commercial use and integration with closed-source projects.

Limitations & Caveats

The project's "To-Do List" mentions "full-body digital humans" as a future goal, implying current limitations in generating complete body avatars. While compatibility with multiple GPUs is mentioned in the Docker section, the primary inference script assumes a single GPU (CUDA_VISIBLE_DEVICES=0).