DINO-WM addresses zero-shot planning in robotics by integrating world models with pre-trained visual features. It empowers agents to perform planning tasks in novel environments without task-specific fine-tuning, benefiting reinforcement learning and robotics researchers.

How It Works

The core approach leverages DINO's pre-trained visual features to construct world models capable of predicting future states from actions. Planning is then executed using methods like Cross-Entropy Method (CEM), aiming for enhanced sample efficiency and generalization by grounding predictions in robust visual representations.

Quick Start & Requirements

Installation involves cloning the repository and setting up a Conda environment via environment.yaml. Key requirements include Mujoco 2.10 (manual download and ~/.bashrc configuration for LD_LIBRARY_PATH), and NVIDIA drivers for GPU acceleration. Optional installation of PyFlex for deformable environments necessitates Docker (docker-ce, nvidia-docker) and a specific installation script or manual Docker compilation. Further details and pre-trained models are available via the Paper and Code links.

Highlighted Details

• Zero-shot planning via DINO pre-trained visual features.
• Supports planning in PointMaze, PushT, and Wall environments.
• Offers pre-trained world model checkpoints for immediate deployment.

Maintenance & Community

Project contributors include researchers from Meta AI (Yann LeCun, Lerrel Pinto) and NYU. No community channels or roadmap details are provided in the README.

Licensing & Compatibility

The license type and compatibility details are not specified in the provided README.

Limitations & Caveats

The optional PyFlex installation for deformable environments adds complexity via Docker. Mujoco setup requires manual file management and environment variable configuration. The project's maturity level (e.g., alpha/beta) is not indicated.