This repository provides an unofficial implementation of InstantID for ComfyUI, enabling users to control image generation with face and pose references. It's designed for artists, researchers, and power users of Stable Diffusion who want to leverage advanced facial and pose conditioning within the ComfyUI node-based workflow. The primary benefit is enhanced control over character identity and pose in generated images.

How It Works

The implementation integrates InstantID's core functionality into ComfyUI nodes. It supports loading base SDXL models from Hugging Face Hub or locally, along with InsightFace models and specific ID ControlNet and IPAdapter models. A key feature is the ID Prompt_Styler node, which allows users to apply various artistic styles (e.g., Watercolor, Film Noir, Neon) to the generated output, alongside positive and negative prompts. The InstantID Generation node takes face and optional pose images, along with model and style configurations, to produce the final output.

Quick Start & Requirements

• Install: Manual installation involves cloning the repository into ComfyUI's custom_nodes directory and running pip install -r requirements.txt. ComfyUI Manager installation is planned.
• Prerequisites: CUDA 11 is supported with onnxruntime-gpu==1.16.0. For CUDA 12, onnxruntime-gpu==1.17.0 must be installed manually. Requires SDXL base models and specific InstantID/ControlNet/IPAdapter models to be downloaded and placed in designated directories.
• Links:
  • V2.0 Video Demo: https://github.com/ZHO-ZHO-ZHO/ComfyUI-InstantID/assets/140084057/083c9e5e-06a0-4623-b5ac-05f7e85a74f2
  • V1.0 Video Demo: https://github.com/ZHO-ZHO-ZHO/ComfyUI-InstantID/assets/140084057/7295c0d7-1d1b-4044-aea3-8efa67047362
  • Workflows: https://github.com/ZHO-ZHO-ZHO/ComfyUI-InstantID#工作流--workflows

Highlighted Details

• V2.0 adds optional pose reference image support, affecting facial region poses.
• Supports automatic downloading of models from Hugging Face Hub.
• Offers 8 distinct style presets via the ID Prompt_Styler node.
• Claims 14s generation time on A100 with 50 steps (V1.0 benchmark).

Maintenance & Community

• Active development with recent updates (Jan 2024) introducing V2.0 features and bug fixes.
• Community support via QQ group (839821928).
• Author contact: zhozho3965@gmail.com.

Licensing & Compatibility

• The repository does not explicitly state a license.
• Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The pose reference functionality is noted to only affect the facial region, differing from standard OpenPose implementations. Compatibility with CUDA 12 requires manual onnxruntime-gpu installation. The lack of an explicit license may pose restrictions for commercial adoption.