VirtualHome is an interactive platform for simulating complex household activities through programmatic control, targeting researchers and developers in embodied AI. It enables the creation and execution of multi-agent household scenarios, offering rich ground-truth data streams and serving as a versatile environment for training AI agents.

How It Works

Activities are defined by programs (sequences of actions) and graphs (environment states). The platform utilizes two simulators: Unity Simulator for video generation and Evolving Graph for Python-based state evolution. This dual approach allows for both visual rendering of activities and detailed, programmatic tracking of environmental changes, facilitating complex interaction modeling.

Quick Start & Requirements

• Install via pip: pip install virtualhome
• Requires downloading Unity Simulator executables (Linux, macOS, Windows) and placing them in simulation/unity_simulator/.
• Unity Simulator can also be run via Docker.
• Official documentation: http://virtual-home.org/documentation/
• Colab demo: https://colab.research.google.com/github/xavierpuigf/virtualhome/blob/master/demo/unity_demo.ipynb

Highlighted Details

• Supports procedural generation for infinite environment variations.
• Enhanced physics, time management, and lighting for realism.
• Provides OpenAI Gym-like environments for RL agent training.
• Offers ground-truth data streams including segmentation, optical flow, and depth.

Maintenance & Community

• Active development with recent updates (v2.3) including procedural generation and performance enhancements.
• Community support via Discord: https://discord.gg/JmzDYF3
• Key contributors include Xavier Puig, Marko Boben, and Sanja Fidler.

Licensing & Compatibility

• The project appears to be primarily distributed under a permissive license, but specific terms for the Unity executable are not detailed in the README.
• Commercial use and closed-source linking compatibility would require careful review of the Unity executable's license.

Limitations & Caveats

• The Unity Simulator requires downloading separate executables, which may have platform-specific dependencies or licensing.
• Some objects and actions supported by the Evolving Graph simulator are not yet implemented in the Unity Simulator.