evolution-strategies-starter  by openai

Research paper code for distributed evolution strategies

created 8 years ago
1,593 stars

Top 26.8% on sourcepulse

GitHubView on GitHub
Project Summary

This repository provides a distributed implementation of Evolution Strategies (ES) as a scalable alternative to Reinforcement Learning, as detailed in the paper "Evolution Strategies as a Scalable Alternative to Reinforcement Learning." It is targeted at researchers and engineers exploring advanced optimization techniques for complex control problems, offering a robust master-worker architecture for parallel computation.

How It Works

The implementation employs a master-worker architecture. The master node distributes current parameters to multiple worker nodes, which then compute gradients or performance metrics. Workers return these results to the master, enabling iterative parameter updates. This design is advantageous for large-scale parallelization, allowing efficient exploration of the parameter space.

Quick Start & Requirements

  • Install/Run: Build custom AMIs using scripts/packer.json and launch experiments with scripts/launch.py.
  • Prerequisites: AWS account, Mujoco (user-provided license and binaries in scripts/dependency.sh), Packer.
  • Setup: Building AMIs and configuring launch scripts requires manual steps.

Highlighted Details

  • Master-worker architecture for distributed computation.
  • Resilient to worker termination, suitable for spot instances.
  • Code used for the humanoid scaling experiment in the cited paper.

Maintenance & Community

This project is archived and no longer actively maintained or updated.

Licensing & Compatibility

The repository's license is not explicitly stated in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project is archived and provided as-is, with no expected updates. It requires a user-provided Mujoco license and manual AMI building, indicating a significant setup overhead and potential compatibility issues with modern environments.

Health Check
Last commit

5 years ago

Responsiveness

1 day

Pull Requests (30d)
0
Issues (30d)
0
Star History
9 stars in the last 90 days

Explore Similar Projects

Starred by Aravind Srinivas Aravind Srinivas(Cofounder of Perplexity), Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and
8 more.

higgsfield by higgsfield-ai

0.3%
3k
ML framework for large model training and GPU orchestration
created 7 years ago
updated 1 year ago
Starred by Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), Nathan Lambert Nathan Lambert(AI Researcher at AI2), and
1 more.

tianshou by thu-ml

0.1%
9k
PyTorch RL library for algorithm development and application
created 7 years ago
updated 1 day ago
Feedback? Help us improve.