RLgraph provides a modular computation graph framework for defining, prototyping, and executing deep reinforcement learning algorithms. It targets researchers and practitioners seeking a unified interface for both static (TensorFlow) and dynamic (PyTorch) graph execution, enabling seamless transition from prototype to large-scale distributed training.

How It Works

RLgraph separates graph definition, compilation, and execution, allowing for multiple distributed backends and device strategies without altering agent definitions. This modularity is achieved through a novel component concept for assembling ML models and a well-defined API for agents. This design facilitates efficient prototyping and scalable deployment.

Quick Start & Requirements

• Install: pip install rlgraph
• Additional dependencies for Ray: pip install rlgraph[ray]
• For tests: pip install gym[all]
• Configuration: A ~/.rlgraph/rlgraph.json file controls backend settings (default: TensorFlow).
• Example usage: Scripts for Ape-X on ALE with Ray and DQN on CartPole are available in the examples folder.
• Documentation: readthedocs

Highlighted Details

• Supports multiple RL algorithms including DQN variants, Ape-X, IMPALA, PPO, and SAC.
• Offers distributed execution capabilities via Ray, with examples for multi-GPU Ape-X and distributed TF IMPALA.
• Implements SingleThreadedWorker for high-performance environment vectorization and RayWorker for Ray actor tasks.
• Includes an extensive test suite, though PyTorch compatibility coverage is not yet full.

Maintenance & Community

• Version 0.4.0 is alpha; core engine is substantially complete.
• Contributions and improvements can be discussed by creating an issue.
• A citation is provided for research use.

Licensing & Compatibility

• License: Not explicitly stated in the README.
• Compatibility: Primarily targets TensorFlow and PyTorch (1.0+). PyTorch backend device handling is incomplete.

Limitations & Caveats

The project is in alpha status (v0.4.0). The PyTorch backend has incomplete device handling and does not yet have full test compatibility with TensorFlow.