This project enables training AI agents for custom multiplayer environments using self-play reinforcement learning, specifically Proximal Policy Optimization (PPO). It's designed for researchers and developers interested in multi-agent AI and game development, offering a structured approach to evolving AI opponents.

How It Works

The core innovation is a wrapper that transforms multiplayer environments into single-player ones for PPO. It manages opponent turn-taking and delays reward signals until all players have acted. This creates a constantly evolving training landscape where new policy versions are integrated into a network bank, allowing agents to learn against increasingly sophisticated versions of themselves.

Quick Start & Requirements

• Installation: Clone the repository, build the Docker image (docker-compose up -d), and install a specific environment (e.g., bash ./scripts/install_env.sh sushigo).
• Prerequisites: Docker and Docker Compose.
• Resources: Training can be parallelized using MPI (mpirun -np 10 python3 train.py -e sushigo).
• Links: Project GitHub, Blog Post

Highlighted Details

• Implements PPO from Stable Baselines.
• Supports custom environments with specific Gym-like methods (step, reset, render, observation, legal_actions).
• Includes test.py for playing against trained agents or baselines and train.py for self-play training.
• TensorBoard integration for monitoring training progress.

Maintenance & Community

• The project is maintained by David Foster (@davidADSP).
• Contributions are welcome via pull requests.
• Roadmap and known issues are tracked in open issues.

Licensing & Compatibility

• Distributed under GPL-3.0.
• GPL-3.0 is a strong copyleft license, potentially restricting integration with closed-source commercial applications.

Limitations & Caveats

The documentation for test.py and train.py command-line arguments is noted as incomplete, with further documentation planned.