This repository provides ComfyUI custom nodes for integrating Tencent's HunyuanVideo model, enabling advanced video generation capabilities. It targets ComfyUI users seeking to leverage state-of-the-art text-to-video and image-to-video generation, including experimental features like Image Prompting (IP2V).

How It Works

The nodes wrap the HunyuanVideo model, offering compatibility with official scaled fp8 weights and various LoRAs. A key innovation is the ability to use native ComfyUI text embeds, bypassing potential issues with the model's original text encoder. It also integrates with the LLava-llama-3-8b-v1_1-transformers model for image-based prompting, allowing images to condition video generation by extracting concepts and styles.

Quick Start & Requirements

• Install nodes via ComfyUI Manager or by cloning the repository into the custom_nodes directory.
• Requires ComfyUI, Python, and PyTorch. Specific dependencies like CUDA 12.x are recommended for optimal performance, especially with fp8 weights.
• Model weights and tokenizers need to be downloaded separately or via auto-download mechanisms to ComfyUI/models/ subdirectories.

Highlighted Details

• Supports official scaled fp8 weights for improved quality.
• Experimental Image Prompting (IP2V) using LLava-llama-3-8b-v1_1-transformers.
• Includes nodes for official I2V, fixed version 2, and Dashtoon keyframe LoRAs.
• Offers a workaround for text encoding using native ComfyUI text embeds.
• Experimental support for Scaled Dot Product Attention (SDPA).

Maintenance & Community

The project is maintained by kijai. Links to example workflows and model repositories are provided within the README.

Licensing & Compatibility

The repository itself appears to be under a permissive license, but the underlying HunyuanVideo model's license should be consulted for commercial use. Compatibility with closed-source linking depends on the HunyuanVideo model's terms.

Limitations & Caveats

Native HunyuanVideo features like context windowing, direct image embed support through IP2V, and manual memory management are noted as missing from this wrapper. Memory usage is significant, potentially limiting high resolutions or frame counts even on 24GB VRAM.