This library provides a high-performance framework for reinforcement learning, targeting researchers and engineers working with complex simulation environments and advanced RL algorithms. It offers GPU-accelerated training pipelines and supports a wide range of algorithms and environments, enabling faster experimentation and deployment.

How It Works

The framework is built with PyTorch and supports both end-to-end GPU acceleration via NVIDIA Isaac Gym and Brax, and CPU-based environments using Ray or EnvPool. It implements various RL algorithms including PPO (with asymmetric actor-critic variants), SAC, and Rainbow DQN. Key features include support for masked actions, multi-agent training with decentralized and centralized critics, and self-play.

Quick Start & Requirements

• Installation: pip install rl-games
• Recommended: PyTorch 2.2+ with CUDA 12.1+ for maximum performance.
• Optional: pip install envpool or pip install ray for CPU environments. Additional gym packages (gym[mujoco], gym[atari], gym[box2d]) and opencv-python may be needed for specific environments.
• Isaac Gym: Requires separate download and installation from NVIDIA.
• Colab Notebooks: Available for quick exploration of Mujoco, EnvPool, and Brax training.

Highlighted Details

• End-to-end GPU accelerated training with NVIDIA Isaac Gym and Brax.
• High performance via EnvPool, offering 3-4x speedup over Ray for Atari.
• Supports ONNX export for discrete and continuous action spaces, including LSTM policies.
• Integrated experiment tracking with Weights & Biases.

Maintenance & Community

• Active development with regular releases and bug fixes.
• Discord channel available: https://discord.gg/hnYRq7DsQh

Licensing & Compatibility

• The repository does not explicitly state a license in the provided README.

Limitations & Caveats

• Older YAML configurations (pre-1.1.0) are not compatible and require updates to parameter names (steps_num to horizon_length, lr_threshold to kl_threshold).
• Running single environments with Isaac Gym may cause crashes; using at least two parallel environments is recommended.