This repository provides custom nodes for ComfyUI, enabling users to integrate and utilize the FLUX.1 diffusion model and its associated ControlNet and LoRA checkpoints. It targets users of Stable Diffusion and ComfyUI who want to leverage advanced generative AI capabilities with specific artistic styles and control mechanisms.

How It Works

The project extends ComfyUI with custom nodes for loading and applying FLUX.1 models, including specialized ControlNet models (Canny, Depth, HED) and LoRAs (Realism, Furry, IP Adapter). It facilitates model management by automatically creating directories for LoRAs and ControlNets within the ComfyUI structure. The integration of IP Adapter allows for image-based conditioning, enhancing control over generated outputs.

Quick Start & Requirements

• Install by cloning the repo into ComfyUI/custom_nodes/x-flux-compyui.
• Run python setup.py within the cloned directory.
• For ControlNet functionality, install comfyui_controlnet_aux from https://github.com/Fannovel16/comfyui_controlnet_aux.
• For low VRAM usage (12GB), follow instructions from https://github.com/city96/ComfyUI-GGUF, use flux1-dev-Q4_0.gguf, and launch ComfyUI with specific parameters (--lowvram, --preview-method auto, --use-split-cross-attention).
• IP Adapter requires downloading Clip-L model.safetensors and the project's IPAdapter models into specific ComfyUI directories.
• Example workflows are available in the workflows folder.

Highlighted Details

• Supports FLUX.1 diffusion model with specialized ControlNet and LoRA checkpoints.
• Offers low VRAM mode for 12GB GPUs.
• Integrates IP Adapter for image-conditioned generation.
• Automatic model directory creation for LoRAs and ControlNets.

Maintenance & Community

No specific contributors, sponsorships, or community links (Discord/Slack) are mentioned in the README.

Licensing & Compatibility

The README does not specify a license. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

The IP Adapter functionality is currently in beta, with no guarantee of immediate good results and potential need for multiple attempts. The project does not specify a license, which may impact commercial adoption.