comfyUI-Realtime-Lora  by shootthesound

Realtime LoRA toolkit for ComfyUI

Created 8 months ago
538 stars

Top 58.3% on SourcePulse

GitHubView on GitHub
Project Summary

This ComfyUI node enables direct LoRA training, analysis, and selective loading within the UI. It targets creators seeking rapid LoRA prototyping and fine-grained control over model fine-tuning without leaving their workflow, streamlining the LoRA development and application process.

How It Works

It unifies three training backends (sd-scripts, Musubi Tuner, AI-Toolkit) for a single interface supporting SDXL, SD 1.5, FLUX, Z-Image, Qwen, and Wan models. Advanced analysis tools identify LoRA block-level impact, complemented by selective loaders for per-block control and strength adjustment, offering precise output manipulation.

Quick Start & Requirements

  • Installation: Clone into ComfyUI custom_nodes.
  • Prerequisites: Python 3.10-3.12 recommended. Training requires separate installation of sd-scripts, Musubi Tuner, or AI-Toolkit backends and specific model downloads. RTX 50-series GPUs need PyTorch 2.7+/CUDA 12.8 via a community installer.
  • Links: Backend repos: kohya-ss/sd-scripts, kohya-ss/musubi-tuner, ostris/ai-toolkit. RTX 50-series installer: omgitsgb/ostris-ai-toolkit-50gpu-installer.

Highlighted Details

  • Real-time LoRA training and editing directly within ComfyUI.
  • Broad model support: SDXL, SD 1.5, FLUX, Z-Image, Qwen, Wan 2.2.
  • LoRA Analyzer visualizes block impact (0-100%); Selective Loaders offer per-block control and strength adjustment.
  • Claims rapid training times, e.g., ~2 minutes for SD 1.5.

Maintenance & Community

Maintained by Peter Neill (ShootTheSound.com), aiming to democratize LoRA training for creators. Feedback via GitHub issues is welcome. Author accepts support via donations. No specific community channels listed.

Licensing & Compatibility

MIT license, generally permitting commercial use, modification, and distribution.

Limitations & Caveats

Training requires separate backend installations and model downloads. AI-Toolkit on RTX 50-series needs a specialized PyTorch/CUDA setup. Python 3.13 is not recommended. Certain model variants (e.g., Qwen fp8) are unsuitable for training.

Health Check
Last Commit

2 months ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
0 stars in the last 30 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), Jeff Hammerbacher Jeff Hammerbacher(Cofounder of Cloudera), and
5 more.

ai-toolkit by ostris

0%
11k
Training toolkit for finetuning diffusion models
Created 3 years ago
Updated 4 days ago
Feedback? Help us improve.