Machin is a PyTorch-based reinforcement learning library offering a comprehensive suite of single-agent, multi-agent, and massively parallel algorithms. It targets researchers and practitioners seeking a readable, reusable, and extendable framework for developing and deploying RL solutions, with features like automatic configuration and distributed training support.

How It Works

Machin is built on PyTorch, leveraging its flexibility for implementing diverse RL algorithms. It emphasizes minimal abstractions and clear implementations, making algorithms easy to understand and adapt. The library supports distributed training through a powerful RPC API, enabling complex parallel execution pools and automatic model assignment for high-performance algorithms like A3C, APEX, and IMPALA.

Quick Start & Requirements

• Install via pip: pip install machin
• Requires Python >= 3.6 and PyTorch >= 1.6.0.
• Limited functionality on non-Linux platforms, particularly for distributed algorithms. Test scripts are provided for Windows, Linux, and macOS.
• Official documentation: See here
• Examples: examples

Highlighted Details

• Supports a wide range of algorithms including DQN, DDPG, A2C, PPO, SAC, MADDPG, A3C, APEX, IMPALA, and imitation learning methods.
• Features automatic configuration generation and experiment launching via machin.auto.
• Designed for readability, reusability, and extensibility, similar to PyTorch's philosophy.
• Implements enhancements like Prioritized Experience Replay (PER) and Generalized Advantage Estimation (GAE).
• Claims reproducible training with a test framework that verifies target scores for each release.

Maintenance & Community

• Primarily maintained by Muhan Li.
• Roadmap available at Roadmap.
• Issue reporting via markdown templates in .github/ISSUE_TEMPLATE.

Licensing & Compatibility

• License is not explicitly stated in the README.
• Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The README notes that only a fraction of all functions is supported on platforms other than Linux, specifically impacting distributed algorithms. Reproducibility tests are not guaranteed to match original papers due to variations in environment setups.