This repository provides the environments and training scripts for the paper "Quantifying Generalization in Reinforcement Learning." It is targeted at researchers and practitioners in reinforcement learning who need to reproduce or extend the CoinRun experiments, offering a standardized benchmark for evaluating generalization capabilities.

How It Works

The project implements custom game environments for reinforcement learning agents, specifically designed to test generalization. It utilizes a procedurally generated maze-like structure with varying difficulty and visual elements. The core training mechanism employs Proximal Policy Optimization (PPO), a popular and effective on-policy reinforcement learning algorithm, integrated with MPI for parallel training.

Quick Start & Requirements

• Install: pip install -e . after cloning the repository and installing dependencies.
• Prerequisites: Python 3.6, TensorFlow 1.12.0 (or tensorflow-gpu), mpich, build-essential, qt5-default, pkg-config (Linux) or qt, open-mpi, pkg-config, git (Mac).
• Setup: Requires compilation of the environment upon first import. A Dockerfile is provided.
• Links: Blog Post, Paper

Highlighted Details

• Codebase for the CoinRun RL environment and associated research.
• Supports training agents using PPO with MPI for parallelization.
• Includes scripts for interactive play, training, and evaluation.
• Offers options for training on specific level sets or continuing from checkpoints.

Maintenance & Community

• Status: Archived; no updates expected.
• Authors: Karl Cobbe, Oleg Klimov, Chris Hesse, Taehoon Kim, John Schulman (OpenAI).

Licensing & Compatibility

• License: Not explicitly stated in the README, but asset licenses are in LICENSES. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project is archived and will not receive updates. It requires specific, older versions of TensorFlow (1.12.0) and Python (3.6), which may pose compatibility challenges with modern systems. The environment compilation process might require manual adjustments to the Makefile on some systems.