This repository provides custom nodes for ComfyUI, enabling users to integrate ELLA (Equip Diffusion Models with LLM for Enhanced Semantic Alignment) into their Stable Diffusion workflows. It targets users of ComfyUI seeking improved semantic alignment and control over image generation, particularly for tasks involving complex prompts, LoRA trigger words, and ControlNet.

How It Works

ELLA leverages a Timestep-Aware Semantic Connector (TSC) to dynamically adapt semantic features across sampling timesteps. This approach enhances alignment between text prompts and generated images. The nodes facilitate the integration of ELLA's conditioning, requiring careful connection to ComfyUI's Conditioning (Concat) node for features like LoRA trigger words to function correctly.

Quick Start & Requirements

• Install by cloning the repository into ComfyUI/custom_nodes/.
• Requires the latest version of ComfyUI.
• Install dependencies via pip install -r requirements.txt within the cloned directory.
• Models (ella-sd1.5-tsc-t5xl.safetensors and FLAN-T5 XL text encoder) must be placed in ComfyUI/models/ella/ and ComfyUI/models/ella_encoder/ respectively.

Highlighted Details

• Supports LoRA trigger words via CLIP conditioning concatenation.
• Works with ControlNet.
• Introduces an ELLA Text Encode node for simplified workflow integration.
• Allows for ELLA-only positive conditioning.
• Compatible with AYS for enhanced visual details and text alignment.

Maintenance & Community

The project is maintained by TencentQQGYLab with contributions from JettHu, budui, kijai, and huagetai.

Licensing & Compatibility

The repository does not explicitly state a license. Compatibility with commercial or closed-source projects is not specified.

Limitations & Caveats

The README notes that ELLA is not implemented for 'Half' precision (see issue #12). Users may encounter dark image generations with AYS + Ella, potentially requiring adjustments to add_noise and CFG scale in SamplerCustom nodes (see issue #39). Prompt weighting support is listed as a future TODO.