This repository provides a PyTorch re-implementation of MiniT2I, a minimalist text-to-image generation model. It addresses the need for a straightforward, direct-RGB generation approach by training a pixel-space denoiser with flow matching, conditioned on frozen text tokens. This project is suitable for researchers and engineers seeking a reproducible baseline for text-to-image synthesis, offering capabilities for inference, LoRA adaptation, and full training.

How It Works

MiniT2I employs a deliberately plain recipe, training a pixel-space MM-JiT denoiser using flow matching. It leverages frozen FLAN-T5-Large text tokens for conditioning, eschewing complex components like image tokenizers, cascaded generation, reinforcement learning stages, or auxiliary losses. This minimalist design simplifies the architecture and training process, focusing on direct image synthesis.

Quick Start & Requirements

Installation involves cloning the repository and installing dependencies:

git clone https://github.com/Hope7Happiness/minit2i-torch.git
cd minit2i-torch
python -m pip install -r requirements.txt

Core inference dependencies include torch, diffusers, transformers, safetensors, pillow, and huggingface_hub. Training and evaluation require configuring local paths for datasets, checkpoints, and outputs in mini_t2i/settings.py. A CUDA-enabled PyTorch build is necessary. A Colab demo is available for quick inference testing.

Highlighted Details

• Offers two model variants: MiniT2I-B/16 (258M parameters) and MiniT2I-L/16 (912M parameters), both using a 341M parameter text encoder.
• Achieves competitive results, with MiniT2I-L/16 scoring 0.878 on GenEval and 84.9 on DPG-Bench.
• Provides integrated support for Hugging Face Diffusers inference, LoRA adaptation, and comprehensive training/evaluation pipelines (GenEval, DPG-Bench, FID).
• Utilizes public datasets for training, including CC12M for pretraining and a 120K mix for fine-tuning.

Maintenance & Community

The project acknowledges contributions from the JAX implementation and various open-source libraries. Specific community channels (e.g., Discord, Slack) or active development roadmaps are not detailed in the README.

Licensing & Compatibility

The specific open-source license for this repository is not explicitly stated in the provided README. Users should verify licensing terms before commercial use or integration into closed-source projects. Compatibility is high within the Hugging Face ecosystem due to its reliance on Diffusers and Transformers.

Limitations & Caveats

Users must manually download and prepare training datasets and evaluation assets, configuring local paths accordingly. Full training configurations for the B/16 and L/16 models are not included in this PyTorch repository; users are directed to the JAX implementation for those details. The license is not specified, which may pose a barrier for certain adoption scenarios.