comfyUI-Realtime-Lora  by shootthesound

Realtime LoRA toolkit for ComfyUI

Created 1 month ago
340 stars

Top 81.2% on SourcePulse

GitHubView on GitHub
Project Summary

This ComfyUI node enables direct LoRA training, analysis, and selective loading within the UI. It targets creators seeking rapid LoRA prototyping and fine-grained control over model fine-tuning without leaving their workflow, streamlining the LoRA development and application process.

How It Works

It unifies three training backends (sd-scripts, Musubi Tuner, AI-Toolkit) for a single interface supporting SDXL, SD 1.5, FLUX, Z-Image, Qwen, and Wan models. Advanced analysis tools identify LoRA block-level impact, complemented by selective loaders for per-block control and strength adjustment, offering precise output manipulation.

Quick Start & Requirements

  • Installation: Clone into ComfyUI custom_nodes.
  • Prerequisites: Python 3.10-3.12 recommended. Training requires separate installation of sd-scripts, Musubi Tuner, or AI-Toolkit backends and specific model downloads. RTX 50-series GPUs need PyTorch 2.7+/CUDA 12.8 via a community installer.
  • Links: Backend repos: kohya-ss/sd-scripts, kohya-ss/musubi-tuner, ostris/ai-toolkit. RTX 50-series installer: omgitsgb/ostris-ai-toolkit-50gpu-installer.

Highlighted Details

  • Real-time LoRA training and editing directly within ComfyUI.
  • Broad model support: SDXL, SD 1.5, FLUX, Z-Image, Qwen, Wan 2.2.
  • LoRA Analyzer visualizes block impact (0-100%); Selective Loaders offer per-block control and strength adjustment.
  • Claims rapid training times, e.g., ~2 minutes for SD 1.5.

Maintenance & Community

Maintained by Peter Neill (ShootTheSound.com), aiming to democratize LoRA training for creators. Feedback via GitHub issues is welcome. Author accepts support via donations. No specific community channels listed.

Licensing & Compatibility

MIT license, generally permitting commercial use, modification, and distribution.

Limitations & Caveats

Training requires separate backend installations and model downloads. AI-Toolkit on RTX 50-series needs a specialized PyTorch/CUDA setup. Python 3.13 is not recommended. Certain model variants (e.g., Qwen fp8) are unsuitable for training.

Health Check
Last Commit

2 weeks ago

Responsiveness

Inactive

Pull Requests (30d)
2
Issues (30d)
23
Star History
184 stars in the last 30 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), Jeff Hammerbacher Jeff Hammerbacher(Cofounder of Cloudera), and
5 more.

ai-toolkit by ostris

1.3%
9k
Training toolkit for finetuning diffusion models
Created 2 years ago
Updated 1 week ago
Feedback? Help us improve.