This repository provides the official implementation of Multi-Agent Transformer (MAT), a novel neural network designed for cooperative Multi-Agent Reinforcement Learning (MARL). MAT bridges MARL and sequence modeling by casting MARL problems into a Transformer-based encoder-decoder architecture, enabling the application of powerful sequence models to MARL challenges. It is an online RL method trained through trial and error, offering superior performance and generalization on benchmarks like StarCraftII, Multi-Agent MuJoCo, and Google Research Football.

How It Works

MAT frames cooperative MARL as a sequence modeling problem, leveraging an encoder-decoder architecture. It utilizes the multi-agent advantage decomposition theorem to achieve linear time complexity for multi-agent scenarios, ensuring monotonic performance improvements. This approach differs from offline methods by employing online reinforcement learning, allowing for adaptation and learning through interaction.

Quick Start & Requirements

• Installation: pip install -r requirements.txt
• Prerequisites:
  • Multi-agent MuJoCo: Requires setup of mujoco-py and multiagent_mujoco, with specific LD_LIBRARY_PATH and LD_PRELOAD environment variables.
  • StarCraft II & SMAC: Run bash install_sc2.sh or manual installation.
  • Google Research Football: Follow instructions at google-research/football.
  • Bi-DexHands: Follow instructions at PKU-MARL/DexterousHands.
• Running Experiments: Execute scripts in the scripts folder (e.g., ./train_mujoco.sh) with algo="mat" or algo="mat_dec". Configuration can be modified in shell files or config.py.
• More Info: Multi-Agent Transformer Site

Highlighted Details

• Outperforms rivals on cooperative MARL benchmarks, demonstrating strong modeling for homogeneous agents.
• Shows advantages in robot control for heterogeneous agents on Multi-Agent Mujoco and Bimanual Dexterous Hands tasks.
• Exhibits excellent few-shot learning capabilities and strong generalization on SMAC tasks.
• Achieves superior performance on Google Research Football tasks compared to MAPPO and HAPPO.

Maintenance & Community

The project is associated with PKU-MARL. Further community or maintenance details are not explicitly provided in the README.

Licensing & Compatibility

The README does not explicitly state the license. The citation format suggests it is based on academic research. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The README does not detail specific limitations, unsupported platforms, or known bugs. The setup for certain environments (MuJoCo, StarCraft II) involves multiple external dependencies and specific configuration steps.