LivePortrait provides an efficient PyTorch implementation for animating portraits using stitching and retargeting control. It enables users to bring static images or videos to life by animating them with a driving video or audio, targeting researchers, developers, and content creators interested in AI-powered video synthesis and manipulation.

How It Works

The project leverages a combination of stitching and retargeting techniques to achieve realistic portrait animation. It likely employs advanced neural network architectures for feature extraction, motion estimation, and image synthesis, allowing for precise control over facial expressions and head movements based on the driving input.

Quick Start & Requirements

• Installation: Clone the repository, create a conda environment (conda create -n LivePortrait python=3.10), activate it (conda activate LivePortrait), and install dependencies (pip install -r requirements.txt).
• Prerequisites: Git, conda, FFmpeg. PyTorch installation requires matching CUDA versions (e.g., CUDA 11.8 for PyTorch 2.3.0). macOS users with Apple Silicon should use requirements_macOS.txt. Animals mode requires building UniPose/ops.
• Pretrained Weights: Download from HuggingFace (huggingface-cli download KwaiVGI/LivePortrait --local-dir pretrained_weights) or alternative sources.
• Inference: Run python inference.py for humans or python inference_animals.py for animals. A Gradio interface is available via python app.py or python app_animals.py.
• Resources: Requires NVIDIA GPU for optimal performance, especially for the Animals model. macOS with Apple Silicon may experience significantly slower inference.

Highlighted Details

• Supports both image and video input for source portraits.
• Offers portrait video editing (v2v) capabilities.
• Includes an optional torch.compile flag for up to 30% faster inference on supported platforms.
• Provides a Gradio interface for user-friendly interaction.
• Animals mode is available but has stricter hardware requirements and is not tested on macOS.

Maintenance & Community

The project is actively updated, with recent changes including Windows installer updates, image-driven mode, regional control, and macOS support. A vibrant community contributes extensions for Stable Diffusion WebUI, ComfyUI nodes, and real-time applications. Resources include HuggingFace Spaces, Colab notebooks, and video tutorials.

Licensing & Compatibility

The repository is released under a permissive license, allowing for commercial use and integration into closed-source projects.

Limitations & Caveats

The Animals mode is only tested on Linux with NVIDIA GPUs and requires custom dependency compilation. macOS with Apple Silicon may face performance limitations and lack support for the Animals mode. Some higher CUDA versions on Windows might cause stability issues.