GibsonEnv is a simulator designed for embodied AI research, providing realistic, real-world environments for training agents in perception and sensorimotor control. It addresses the challenges of real-world robot fragility and slow learning by offering a simulated, physics-enabled space that facilitates efficient training and aims for seamless transfer to physical systems.

How It Works

GibsonEnv integrates a physics engine (Bulletphysics) to simulate agent embodiment and environmental interactions. Its core innovation lies in virtualizing real-world spaces, capturing semantic complexity, and incorporating a "Goggles" function for domain adaptation. This function acts as a learned inverse mapping to bridge the sim-to-real gap by transforming real-world camera inputs to match simulated ones, enabling more robust transfer learning.

Quick Start & Requirements

• Docker (Recommended):
  • Requires Ubuntu 16.04, NVIDIA GPU (>6GB VRAM), NVIDIA driver (>=384), CUDA (>=9.0), CuDNN (>=v7).
  • docker pull xf1280/gibson:0.3.1
  • docker run --runtime=nvidia -ti --rm -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v :/root/mount/gibson/gibson/assets/dataset xf1280/gibson:0.3.1
  • Dataset download required separately: https://storage.googleapis.com/gibson_scenes/dataset.tar.gz
• Build from Source:
  • Requires Ubuntu (>=14.04), NVIDIA GPU (>6GB VRAM), NVIDIA driver (>=375), CUDA (>=8.0), CuDNN (>=v5).
  • Dependencies include libglew-dev, libglm-dev, libassimp-dev, cmake, python3.5, pytorch, tensorflow==1.3.
  • git clone https://github.com/StanfordVL/GibsonEnv.git
  • cd GibsonEnv && ./download.sh && ./build.sh build_local && pip install -e .
• Official Docs: http://gibsonenv.stanford.edu/

Highlighted Details

• Features 572 real-world spaces and 1440 floors, with a subset included by default.
• Supports various robotic agents (Ant, Humanoid, Husky, TurtleBot) with different controllers.
• Includes a web UI for remote access and headless server operation.
• Offers semantic scene understanding capabilities with datasets like Stanford 2D-3D-Semantics and Matterport 3D.

Maintenance & Community

• The project is from Stanford University.
• Encourages bug reports and community developments.
• Changelog available.

Licensing & Compatibility

• The README does not explicitly state a license. The code is distributed by Stanford University. Commercial use and linking compatibility are not specified.

Limitations & Caveats

• The project is at release 0.3.1, suggesting potential for ongoing development and breaking changes.
• Building from source requires specific older versions of Python (3.5) and deep learning libraries (PyTorch, TensorFlow 1.3), which may pose compatibility challenges with modern systems.
• The "Goggles" function requires a camera with a depth sensor for optimal use, though an RGB-only example is provided.