This repository provides an implementation of various Evolution Strategies (ES) algorithms, including GA, Population-based REINFORCE, CMA-ES, and OpenAI's ES, with a common interface. It's designed for researchers and practitioners in reinforcement learning and evolutionary computation who need a flexible tool for optimizing policies or controllers, particularly in simulated environments.

How It Works

The core of the tool is the EvolutionStrategy class, which abstracts the ask-tell interface common to ES algorithms. Users provide candidate solutions via solver.ask(), evaluate them to obtain rewards, and then feed these rewards back using solver.tell(). This approach allows for easy integration of different ES variants and simplifies the process of experimenting with various optimization strategies. The library supports parallel processing via mpi4py for distributed training.

Quick Start & Requirements

• Install: pip install -r requirements.txt (or manually install dependencies).
• Prerequisites: NumPy (tested with 1.13.3), OpenAI Gym (tested with 0.9.4), cma (2+), PyBullet (tested with 1.6.3), Python 3, mpi4py. Some environments may require Box2D.
• Setup: MPI setup on Windows requires MS-MPI and Visual Studio C++ compiler.
• Docs: Evolving Stable Strategies

Highlighted Details

• Implements GA, Population-based REINFORCE, CMA-ES (via pycma), and OpenAI ES.
• Supports parallel training using mpi4py for distributed computation.
• Includes self-contained examples like Cartpole Swingup and Slime Volleyball.
• Provides utilities for plotting training progress from .hist.json files.

Maintenance & Community

The project is maintained by hardmaru. The primary reference is a blog post from 2017. No explicit community channels (like Discord/Slack) are mentioned.

Licensing & Compatibility

The repository does not explicitly state a license in the README. This requires clarification for commercial use or integration into closed-source projects.

Limitations & Caveats

The project relies on older versions of dependencies like OpenAI Gym (0.9.4), which may cause compatibility issues with newer Gym APIs. The lack of an explicit license is a significant caveat for adoption.