This repository provides code for reproducing key results from the paper "Improving Variational Inference with Inverse Autoregressive Flow." It targets researchers and practitioners in deep learning and generative modeling interested in advanced variational inference techniques. The primary benefit is enabling replication of state-of-the-art results in variational inference.

How It Works

The project implements Inverse Autoregressive Flows (IAFs) within a variational inference framework. IAFs are a class of generative models that use autoregressive transformations to construct complex probability distributions from simpler ones. This approach allows for more flexible and powerful modeling of latent variables in variational autoencoders, leading to improved likelihood estimates.

Quick Start & Requirements

• Theano Implementation:

  • Install: pip install Theano numpy
  • Prerequisites: Python 2.7+, Theano, CIFAR-10 dataset (with CIFAR10_PATH environment variable).
  • Configuration: Set floatX = float32 in Theano config or prepend THEANO_FLAGS=floatX=float32.
  • Example: python train.py with problem=cifar10 n_z=32 n_h=64 depths=[2,2,2] margs.depth_ar=1 margs.posterior=down_iaf2_NL margs.kl_min=0.25
  • Documentation: Paper

• TensorFlow Implementation:

  • Install: TensorFlow, tqdm.
  • Prerequisites: Python 2.7+, CIFAR10_PATH environment variable.
  • Example (Training): python tf_train.py --logdir <logdir> --hpconfig depth=1,num_blocks=20,kl_min=0.1,learning_rate=0.002,batch_size=32 --num_gpus 8 --mode train
  • Example (Evaluation): python tf_train.py --logdir <logdir> --hpconfig depth=1,num_blocks=20,kl_min=0.1,learning_rate=0.002,batch_size=32 --num_gpus 1 --mode eval_test
  • TensorBoard: tensorboard --logdir <logdir>

Highlighted Details

• Reproduces key results from the IAF paper, including achieving 3.28 bits/dim on CIFAR-10 with the Theano implementation.
• The TensorFlow implementation supports multi-GPU training and evaluation.
• Offers various posterior approximations (e.g., up_diag, up_iaf2_nl, down_iaf2_nl).
• Provides a pre-trained checkpoint for CIFAR-10, achieving 3.15 bits/dim with single-sample evaluation.

Maintenance & Community

• Status: Archived (code provided as-is, no updates expected).
• Authors: Diederik P. Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, Max Welling.

Licensing & Compatibility

• License: Not explicitly stated in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project is archived and will not receive further updates. The Theano implementation requires Python 2.7, which is end-of-life. License details for commercial use are absent.