This repository provides ComfyUI nodes for advanced image generation and upscaling, specifically addressing VRAM limitations. It enables users to create ultra-large images and perform high-resolution upscaling using techniques like Tiled Diffusion, MultiDiffusion, and Mixture of Diffusers, along with an optimized VAE.

How It Works

The core of this project lies in its implementation of tiling strategies for diffusion models. Tiled Diffusion breaks down large images into smaller tiles, processing them sequentially to manage VRAM. MultiDiffusion and Mixture of Diffusers further enhance this by allowing the fusion of multiple diffusion paths or models, enabling more complex and controlled generation. The optimized Tiled VAE efficiently handles the encoding and decoding of these tiles, with a "fast" mode that minimizes VRAM-to-RAM transfers by reusing GroupNorm parameters across tiles.

Quick Start & Requirements

• Install via ComfyUI's custom node manager or by cloning the repository into the custom_nodes directory.
• Requires ComfyUI and compatible Stable Diffusion models (SD1.x, SD2.x, SDXL, SD3, FLUX).
• ControlNet support is available.
• Official documentation and example workflows are linked in the README.

Highlighted Details

• Supports SD1.x, SD2.x, SDXL, SD3, and FLUX models.
• Includes ControlNet integration.
• Features an optimized Tiled VAE for efficient encoding/decoding.
• Offers advanced tiling strategies like MultiDiffusion and Mixture of Diffusers.

Maintenance & Community

The project acknowledges contributions from the community and cites research papers for its core techniques. Links to community resources like Discord or Slack are not explicitly provided in the README.

Licensing & Compatibility

The implementation of MultiDiffusion, Mixture of Diffusers, and Tiled VAE code is under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Other components are licensed under GPLv3. This combination may restrict commercial use or linking with closed-source projects due to the non-commercial and copyleft clauses.

Limitations & Caveats

StableSR and Tiled Noise Inversion are not supported. The "fast" mode for the Tiled VAE may result in slightly higher contrast and brightness in generated images. The SpotDiffusion method is experimental and subject to change.