This repository hosts the 1X World Model Challenge, aimed at accelerating progress in learned simulators for general-purpose robotics. It provides a dataset of over 100 hours of first-person EVE Android robot observations and actions, along with baseline models and evaluation tools, targeting researchers and engineers in robotics and AI.

How It Works

The core of the challenge involves predicting future robot observations using learned world models. The approach utilizes a GENIE-style spatio-temporal transformer and a MAGVIT2 autoencoder to compress images into discrete tokens. This tokenized representation allows for efficient modeling of sequential data, enabling the prediction of future states within a learned simulation environment. The advantage lies in creating end-to-end learned simulators that can significantly speed up robot policy development.

Quick Start & Requirements

• Install dependencies and download data: ./build.sh
• Activate Python environment: source venv/bin/activate
• Requires Python 3.10 or later.
• Official quick-start and dataset details are available on Huggingface.

Highlighted Details

• Three challenges: Compression (predicting tokens), Sampling (generating plausible videos), and Evaluation (ranking policies within a world model).
• Cash prizes are offered for intermediate goals in the Compression and Sampling challenges.
• The dataset consists of 16 first-person images at 2Hz, totaling 8 seconds per sequence.
• Pre-trained GENIE models are provided for baseline comparison.

Maintenance & Community

• The project is associated with 1X Technologies.
• A Discord server is available for community discussion.
• Updates on Phase 2 of the challenge are expected.

Licensing & Compatibility

• The dataset is licensed under Apache 2.0.
• The repository itself does not explicitly state a license, but the dataset license suggests commercial use is permitted.

Limitations & Caveats

• Prizes are not available to individuals in U.S. sanctioned countries.
• The challenge organizers reserve the right to withhold prizes if the spirit of the challenge is violated.
• The loss metric is non-standard due to a factorized probability mass function for image tokens.