This repository provides ControlNet models specifically for the Flux1.dev diffusion model, targeting users who need to generate images from line art or outline sketches. It offers enhanced alignment and expressiveness for various lineart conditions using a dual-stream Transformer architecture, compatible with Flux1.dev's quantized models.

How It Works

The ControlNet utilizes a scalable Transformer module as its backbone, featuring a dual-stream structure. This design improves alignment and expressiveness for lineart and outline inputs without increasing inference time. It's trained for alignment with both T5 and clip-l TextEncoders, ensuring a balance between image conditioning and text prompts.

Quick Start & Requirements

• Download the model from Huggingface: MistoLine_Flux.dev_v1.
• Place the model in ComfyUI\models\TheMisto_model\.
• Run using ComfyUI; an example workflow is provided in the workflow folder.
• Conditioning image dimensions must be divisible by 16.
• Requires TheMisto.ai Flux ControlNet ComfyUI suite.
• Compatible with Flux1.dev's fp16/fp8 and other quantized models.
• Recommended settings: 720px+ resolution, controlnet strength 0.6-0.85, guidance 3.0-5.0, 30+ steps.

Highlighted Details

• ControlNet model parameters: ~1.4B.
• Trained for alignment with T5 and clip-l TextEncoders.
• Compatible with Flux1.dev's fp16/fp8 and quantized models (e.g., flux1-dev-Q4_K_S.gguf).
• Performance is positively correlated with prompt quality; experiment with controlnet_strength.

Maintenance & Community

• Developed by TheMisto.ai Team.
• Community links: Discord (https://discord.gg/fTyDB2CU), X (https://x.com/AiThemisto79359).
• Future product: Misto, a multi-modal AI creative tool.

Licensing & Compatibility

• License: FLUX.1 [dev] Non-Commercial License.
• Usage: Research and educational purposes only; commercial use is prohibited.

Limitations & Caveats

• Not compatible with XLabs loaders and samplers.
• Training requires consumer-grade GPUs (e.g., A100-80GB with bf16) and is computationally expensive; consumer GPUs are unsuitable for training.
• ByteDance 8/16-step distilled models have not been tested.