This repository provides experimental code for reproducing the "Recurrent World Models Facilitate Policy Evolution" paper. It targets researchers and engineers interested in generative models, reinforcement learning, and agent-based simulations, offering a direct implementation of the paper's core concepts.

How It Works

The project implements the World Models framework, which consists of a VAE for compressing observations into a compact latent space, a recurrent model (MDN-RNN) that predicts future latent states and rewards from the current latent state and action, and a controller (e.g., a simple MLP or evolutionary algorithm) that learns to act in the environment based on the world model's predictions. This approach allows agents to learn in a compressed latent space, potentially enabling faster and more efficient exploration and policy learning.

Quick Start & Requirements

• Install: pip install gym==0.9.4 numpy==1.13.3 (specific older versions required).
• Prerequisites: Python 3.x, OpenAI Gym (version 0.9.x).
• Setup: Requires careful installation of specific library versions to ensure compatibility.

Highlighted Details

• Direct implementation of the World Models paper by David Ha and Jürgen Schmidhuber.
• Focuses on reproducing the experiments and methodology described in the paper.
• Includes code for VAE, MDN-RNN, and controller components.

Maintenance & Community

• The repository is primarily an archival implementation of the paper's experiments.
• A more recent TensorFlow 2.2 implementation by @zacwellmer is recommended for updated frameworks and Dockerized reproduction.

Licensing & Compatibility

• License: MIT.
• Compatibility: The MIT license permits commercial use and linking with closed-source projects. However, the strict dependency on older library versions (Gym 0.9.x) may pose integration challenges with modern Python environments.

Limitations & Caveats

This implementation relies on outdated library versions (specifically OpenAI Gym 0.9.x), which may require significant effort to set up and maintain in current development environments. It serves as a direct reproduction of the original paper's experiments rather than a readily usable, modern library.