Arc2Face is a foundation model for generating high-quality, ID-consistent human faces from ArcFace embeddings. It targets researchers and developers in computer vision and generative AI, enabling rapid creation of diverse facial images for a specific identity.

How It Works

Arc2Face builds upon Stable Diffusion v1.5, replacing its text encoder with a custom CLIPTextModelWrapper and fine-tuning the UNet model. This architecture allows it to condition image generation directly on identity embeddings extracted using the ArcFace metric learning model. This approach ensures strong identity preservation while leveraging the generative power of diffusion models.

Quick Start & Requirements

• Install: Create a conda environment (conda create -n arc2face python=3.10) and install requirements (pip install -r requirements.txt).
• Models: Download Arc2Face checkpoints, an ArcFace ONNX model, and the antelopev2 package from HuggingFace, placing them in a models directory.
• Hardware: Requires a CUDA-enabled GPU for inference.
• Demo: A Gradio demo is available on HuggingFace Spaces.
• Docs: Official implementation for ECCV 2024 Oral paper.

Highlighted Details

• Trained on the large-scale WebFace42M dataset for superior ID similarity.
• Supports LCM-LoRA for significantly faster inference (2-4 steps).
• Offers ControlNet integration for pose control.
• ComfyUI and Pinokio implementations are available.

Maintenance & Community

• Accepted to ECCV24 as an oral presentation.
• Active development with recent updates including ComfyUI support and LCM-LoRA integration.
• Community resources include Replicate and Pinokio demos.

Licensing & Compatibility

• The repository does not explicitly state a license. The models are available on HuggingFace.
• Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

• The installation of the ControlNet component, particularly PyTorch3D, is noted as potentially complex and prone to conflicts.
• LCM-LoRA integration offers faster inference at the cost of a slight quality reduction.