This repository provides PyTorch implementations of key Policy Gradient (PG) reinforcement learning algorithms, including REINFORCE, NPG, TRPO, and PPO. It targets researchers and practitioners in reinforcement learning, offering a unified framework for experimenting with and comparing these advanced PG methods on standard benchmarks.

How It Works

The project implements four distinct PG algorithms: Vanilla Policy Gradient, Truncated Natural Policy Gradient, Trust Region Policy Optimization (TRPO), and Proximal Policy Optimization (PPO). It leverages PyTorch for model definition and training. The implementations are designed to be modular, allowing for easy switching between algorithms and hyperparameter tuning. The use of standard RL benchmarks like Mujoco and Unity ml-agents facilitates reproducible research and direct comparison of algorithm performance.

Quick Start & Requirements

• Mujoco:
  • Install: pip install -r requirements.txt (within pg_travel/mujoco)
  • Prerequisites: mujoco-py (requires a license from DeepMind), Python 3.x.
  • Run: python main.py (defaults to PPO on Hopper-v2)
  • Docs: Mujoco-py
• Unity ml-agents:
  • Install: Download prebuilt environments and place in pg_travel/unity/env.
  • Prerequisites: Unity ml-agents, Python 3.x.
  • Run: python main.py --train (within pg_travel/unity)
  • Docs: Unity ml-agents

Highlighted Details

• Implements Vanilla PG, NPG, TRPO, and PPO algorithms.
• Supports both mujoco-py and custom Unity ml-agents environments.
• Includes TensorboardX integration for visualizing training progress.
• Provides options for continuing training from checkpoints and testing pre-trained models.

Maintenance & Community

The repository is maintained by reinforcement-learning-kr. Further community engagement details are not explicitly provided in the README.

Licensing & Compatibility

The repository's license is not explicitly stated in the README. Compatibility for commercial use or closed-source linking would require clarification of the licensing terms.

Limitations & Caveats

The project uses PyTorch v0.4.0, which is an older version and may have compatibility issues with newer PyTorch releases or libraries. Trained agents and Unity ml-agent environment source files are noted as "soon to be available," indicating potential incompleteness.