This repository provides an unofficial PyTorch implementation of BigGAN, designed for researchers and practitioners aiming to train high-fidelity generative models for natural image synthesis. It offers a flexible and extensible codebase for experimenting with BigGAN architectures and training strategies, including gradient accumulation for simulating larger batch sizes on multi-GPU setups.

How It Works

The implementation leverages gradient accumulation to simulate large effective batch sizes, enabling training of BigGAN models with batch sizes up to 2048 on multi-GPU systems without requiring TPU pods. It uses PyTorch's built-in inception network for calculating FID and IS metrics, with notes on differences from TensorFlow implementations and guidance on using TensorFlow for official scores. The code is structured for extensibility, allowing easy modification of components like loss functions, architectures, and attention blocks.

Quick Start & Requirements

• Install: PyTorch (>= 1.0.1), tqdm, numpy, scipy, h5py.
• Data: Requires ImageNet training set. Optional HDF5 preprocessing available via sh scripts/utils/prepare_data.sh.
• Hardware: Recommended 4-8 GPUs (e.g., V100 16GB). Training a full-sized BigGAN on 8xV100 takes ~15 days for 150k iterations.
• Memory: --load_in_mem argument requires ~96GB+ RAM for loading HDF5.
• Docs: Official BigGAN paper

Highlighted Details

• Supports gradient accumulation for large effective batch sizes (e.g., 256x8 for BS2048).
• Includes optimized PyTorch implementations for FID calculation and orthogonal regularization.
• Provides two pre-trained BigGAN checkpoints (128x128 ImageNet).
• Offers scripts for BigGAN-deep, CIFAR, SA-GAN, and SN-GAN.

Maintenance & Community

• Developed by Andy Brock and Alex Andonian.
• Mentions contributions from various sources for specific components.
• Open to issues and pull requests for improvements.

Licensing & Compatibility

• The repository does not explicitly state a license in the README. Users should verify licensing for commercial or closed-source use.

Limitations & Caveats

The provided pre-trained models were not trained with orthogonal regularization, which may affect amenability to truncation. Synchronized BatchNorm and mixed-precision training (Tensor Cores) are noted as desired features that are not yet implemented, with suggestions for using Apex. The BigGAN-deep scripts are considered untested.