EasyPhoto is a Stable Diffusion WebUI plugin for generating personalized AI portraits and digital doppelgangers. It allows users to train a custom model using a small set of their own photos and then generate new images in various styles and scenarios, including video generation and attribute editing.

How It Works

EasyPhoto leverages Stable Diffusion's image-to-image capabilities and LoRA fine-tuning to create a user's digital doppelganger. It preprocesses user images to isolate faces, then fine-tunes a Stable Diffusion model. During inference, it uses a template image, fuses the user's face onto it, and employs ControlNets (like Canny and OpenPose) to guide the generation process, ensuring likeness and stability. A two-stage diffusion process refines the output for higher quality.

Quick Start & Requirements

• Installation: Installable as a plugin for AUTOMATIC1111's Stable Diffusion WebUI via https://github.com/aigc-apps/sd-webui-EasyPhoto. Docker image available: registry.cn-beijing.aliyuncs.com/mybigpai/sd-webui-easyphoto:0.0.3.
• Prerequisites: Requires an existing Stable Diffusion WebUI installation, ControlNet extension (Mikubill/sd-webui-controlnet), and at least 12GB VRAM GPU (16GB recommended for SDXL). Verified environments include Python 3.10, PyTorch 2.0.1, CUDA 11.7, and NVIDIA GPUs (3060 12G, A10 24G, V100 16G, A100 40G). Approximately 60GB disk space is needed.
• Resources: Cloud options include Aliyun DSW, AutoDL, and lanrui-ai. Demo available on ModelScope.

Highlighted Details

• Supports LCM-Lora for accelerated image/video generation (12 steps vs. 50).
• Features Concepts-Sliders for attribute editing and Virtual TryOn.
• Offers SDXL training and inference for high-resolution outputs.
• Provides ComfyUI support and a Diffusers edition.

Maintenance & Community

• Active development with recent updates including LCM-Lora, attribute editing, SDXL, and video inference.
• Community support via DingTalk group (ID: 54095000124).
• Follows the all-contributors specification.

Licensing & Compatibility

• Licensed under the Apache License (Version 2.0).
• Compatible with commercial use and closed-source linking under Apache 2.0 terms.

Limitations & Caveats

• Local installation requires careful environment setup, including specific Python, PyTorch, and CUDA versions.
• Out-of-memory (OOM) errors can occur on lower-spec GPUs; refer to issue #21 for potential fixes.
• Docker image updates may lag behind the GitHub repository.