This project provides a C++ reinforcement learning framework leveraging the PyTorch C++ frontend, targeting developers needing RL capabilities in C++ environments where Python is not viable. It offers an extensible, optimized solution for production use in desktop applications, aiming for minimal user-side setup.

How It Works

CppRl is a direct port of the popular pytorch-a2c-ppo-acktr-gail Python library, retaining its API and core algorithms. It implements A2C and PPO with support for recurrent policies (GRU-based), continuous, and discrete action spaces. An integrated OpenAI Gym server using ZeroMQ facilitates testing against Gym environments.

Quick Start & Requirements

• Install: Build using CMake. Dependencies like Libtorch must be installed separately.
• Prerequisites: Libtorch, CMake.
• Setup: git submodule update --init --recursive followed by CMake build commands.
• Demo: Run launch_gym_server.py and build/example/gym_server.
• Docs: [No explicit link provided in README]

Highlighted Details

• Implements A2C and PPO algorithms.
• Supports recurrent policies (GRU-based).
• Handles both continuous and discrete action spaces.
• Cross-platform compatibility (Windows, Ubuntu tested).

Maintenance & Community

• Based on Ikostikov's pytorch-a2c-ppo-acktr-gail.
• [No specific community links or contributor information provided in README]

Licensing & Compatibility

• [License type not specified in README]
• Suitable for integration into C++ applications.

Limitations & Caveats

The included Gym server and client have performance limitations, especially with image observations, due to inter-process communication overhead and data copying. Windows performance is noted as approximately 75% of Linux performance.