Naifu is a PyTorch Lightning-based framework for training generative models, primarily diffusion models and large language models. It targets researchers and developers looking for a flexible, configuration-driven approach to fine-tune or train models like Stable Diffusion XL, Stable Cascade, Pixart-Alpha, and various LLMs. The framework aims to simplify complex training setups with a unified interface.

How It Works

Naifu leverages PyTorch Lightning for distributed training and efficient checkpointing. It uses YAML configuration files to define training parameters, model architectures, datasets, and optimization strategies. This approach allows users to easily switch between different model types (e.g., SDXL, LLMs) and training techniques (e.g., LyCORIS, DPO, LCM) without modifying core code.

Quick Start & Requirements

• Install:

  git clone --depth 1 https://github.com/mikubill/naifu
  cd naifu && pip install -r requirements.txt
  

• Prerequisites: Python 3.9+. Specific models may require additional libraries (e.g., lycoris_lora, toml, fairscale). GPU with CUDA is highly recommended for practical training.
• Usage: python trainer.py --config <config_file>
• Docs: https://github.com/mikubill/naifu

Highlighted Details

• Supports training for a wide array of models including SDXL, SDXL Refiner, Stable Diffusion 1.x/1.5, Pixart-Alpha, StableCascade, GPT2, and various LLMs (LLaMA, Qwen, Gemma, Mistral).
• Integrates advanced training techniques like LyCORIS, LoRA/QLoRA, Diffusion DPO, and Latent Consistency Models (LCM).
• Offers specialized branches for different model architectures and backends (e.g., sgm, sd3, hydit).
• Includes scripts for data preprocessing, such as encoding images to latents for SDXL.

Maintenance & Community

The main branch is under active development. The repository is maintained by Mikubill. Community links are not explicitly provided in the README.

Licensing & Compatibility

The repository does not explicitly state a license in the README. Users should verify licensing for commercial use or integration into closed-source projects.

Limitations & Caveats

The main branch is under development and subject to change. Some specialized branches (e.g., sd3) are noted as experimental or potentially producing undesired results. Specific model training configurations may require careful tuning of parameters like resolution and beta values.