This repository provides an open-source implementation of DeepMind's Sequential Social Dilemma (SSD) environments, designed for multi-agent reinforcement learning research. It offers game-theoretic scenarios like "Cleanup" and "Harvest" where individual optimal strategies can lead to suboptimal group outcomes, making them valuable for studying cooperation and defection dynamics. The environments are compatible with OpenAI Gym and RLlib.

How It Works

The environments simulate spatially and temporally extended Prisoner's Dilemma-like games. Agents interact within a shared space, with rewards influenced by collective actions. For instance, in "Cleanup," agents must clean a shared resource to enable apple growth, but can exploit the resource while others clean. This design allows for the study of emergent cooperative or exploitative behaviors in multi-agent systems.

Quick Start & Requirements

• Installation: Clone the repository and use conda or venv for environment setup. Install dependencies with pip install social-dilemmas[sb3|rllib|all].
• Prerequisites: Python 3.8.10, git, conda or venv. CUDA/cuDNN compatibility is required if using TensorFlow with GPU acceleration.
• Running Experiments: Use python3 run_scripts/train.py for RLlib training or run_scripts/sb3_train.py for Stable-Baselines3.
• Documentation: Game rules and implementation details can be found in the test folder.

Highlighted Details

• Implements "Cleanup" (public goods dilemma) and "Harvest" (tragedy of the commons) environments.
• Compatible with RLlib and Stable-Baselines3 for multi-agent reinforcement learning.
• Includes scripts for training with PPO and MARL-Baselines3.
• Offers empirical Schelling diagrams illustrating cooperation vs. defection payoffs.

Maintenance & Community

The codebase was initially developed by Eugene Vinitsky and Natasha Jaques, with contributions from Joel Leibo, Antonio Castenada, and Edward Hughes. Support for PettingZoo is provided by Rohan Potdar.

Licensing & Compatibility

The repository does not explicitly state a license in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project is marked as deprecated, with a strong recommendation to use DeepMind's Melting Pot instead. RLlib initialization time can be significant (up to 5 minutes) with more agents or complex models. CUDA/cuDNN version compatibility with TensorFlow can be challenging.