This project provides a Cog wrapper for fine-tuning FLUX.1 image generation models using LoRA, enabling users to create custom image generation models. It also includes inference capabilities for these fine-tuned models, targeting users who want to personalize image generation workflows.

How It Works

The fine-tuner leverages the ostris/ai-toolkit and integrates LoRA for efficient model adaptation. It automates image captioning during training and allows for inference with the resulting LoRA weights. The system supports Weights & Biases integration for experiment tracking and can optionally upload fine-tuned weights to Hugging Face.

Quick Start & Requirements

• Install/Run: Primarily used via Replicate's platform (replicate.com/ostris/flux-dev-lora-trainer). For local development, Cog is required.
• Prerequisites: GPU with sufficient VRAM for training, Python, and Cog. Specific CUDA versions are not explicitly stated but are generally required for GPU acceleration.
• Resources: Training can be resource-intensive, requiring significant GPU memory and time depending on dataset size and training parameters.
• Links:
  • Replicate Trainer: https://replicate.com/ostris/flux-dev-lora-trainer
  • Fine-tuning Guides: https://replicate.com/blog/fine-tune-flux-to-create-images-of-yourself, https://replicate.com/blog/fine-tune-flux-with-an-api

Highlighted Details

• LoRA-based fine-tuning for FLUX.1 models.
• Automatic image captioning during training.
• Inference support for fine-tuned models.
• Weights & Biases integration and optional Hugging Face uploads.

Maintenance & Community

The project is associated with ostris and appears to be actively used on Replicate. Contributing guidelines are available for developers.

Licensing & Compatibility

• License: Apache-2.0 (for this repository code), MIT (for ai-toolkit), FLUX.1 [dev] Non-Commercial License (for FLUX.1 models and their outputs).
• Commercial Use: Fine-tuned FLUX.1 weights and outputs are non-commercial by default but can be used commercially when run on Replicate. Flux Schnell is Apache-2.0 licensed.

Limitations & Caveats

The FLUX.1 models and their fine-tuned weights are subject to a non-commercial license, with commercial use restricted to the Replicate platform. Local commercial use of fine-tuned weights may require separate licensing.