Crafter is an open-world survival game designed as a benchmark for evaluating a wide range of agent capabilities in reinforcement learning. It presents researchers and engineers with a single, computationally efficient environment to test generalization, exploration, representation learning, and long-term reasoning, offering meaningful, semantically-driven achievements for evaluation.

How It Works

Crafter simulates a 2D procedurally generated world where agents must forage, build, and survive against monsters. It focuses on a spectrum of agent abilities by tracking 22 distinct achievements, providing a richer evaluation than simple reward signals. The environment adheres to the OpenAI Gym interface, outputting 64x64 RGB observations and supporting 17 categorical actions.

Quick Start & Requirements

• Install: pip install crafter
• Human Interface: pip install pygame and run python3 -m crafter.run_gui
• Dependencies: Python 3.x
• Documentation: OpenAI Gym interface

Highlighted Details

• Evaluates agents on success rates of 22 achievements, with scores derived from the geometric mean of these rates.
• Offers two modes: CrafterReward-v1 (sparse reward for achievements) and CrafterNoReward-v1 (unsupervised learning focus).
• Human expert performance reaches a score of 50.5%, with RL agents achieving up to ~19.4%.
• Includes example evaluation scripts and baseline scores in JSON format.

Maintenance & Community

• Maintained by Danijar Hafner.
• Questions can be directed via GitHub Issues.

Licensing & Compatibility

• License: Apache License 2.0.
• Compatible with commercial use and closed-source linking.

Limitations & Caveats

The environment is designed for research and may require significant computational resources for training agents to achieve high scores. While stable, it is primarily a benchmarking tool, and its complexity might not suit all RL research needs.