This project provides custom nodes for ComfyUI to enable the use of GGUF-quantized models, specifically targeting transformer/DiT architectures. It aims to allow users to run diffusion models with significantly reduced VRAM requirements on lower-end GPUs, making advanced AI image generation more accessible.

How It Works

The nodes leverage the GGUF format, popularized by llama.cpp, to load and run quantized UNET models. This approach is advantageous because transformer/DiT models are less sensitive to quantization compared to traditional UNETs, enabling substantial VRAM savings through variable bitrate quants. Additionally, a node for loading quantized T5 text encoders is included for further memory optimization.

Quick Start & Requirements

• Install via git clone https://github.com/city96/ComfyUI-GGUF into your ComfyUI custom_nodes folder.
• Install dependency: pip install --upgrade gguf.
• For standalone ComfyUI installations, use .\python_embeded\python.exe -s -m pip install -r .\ComfyUI\custom_nodes\ComfyUI-GGUF\requirements.txt.
• Requires a recent ComfyUI version supporting custom ops.
• MacOS Sequoia users may need PyTorch 2.4.1 due to compatibility issues with newer versions.
• GGUF model files should be placed in ComfyUI/models/unet.

Highlighted Details

• Supports GGUF quantized models for UNETs, including flux1-dev, flux1-schnell, stable-diffusion-3.5-large, and stable-diffusion-3.5-large-turbo.
• Includes experimental LoRA loading support.
• Offers nodes for loading quantized T5 text encoders (e.g., t5_v1.1-xxl GGUF).
• Provides instructions for creating custom GGUF quantizations.

Maintenance & Community

• This project is marked as "very much WIP" (Work In Progress).
• No specific community links or notable contributors are mentioned in the README.

Licensing & Compatibility

• The README does not explicitly state a license. The repository itself is under the MIT license.
• Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project is in an early stage of development ("very much WIP"). Compatibility issues with PyTorch versions on macOS have been noted, and the effectiveness of LoRA loading is experimental.