Habitat-GS is a high-fidelity navigation simulator designed for embodied AI research, extending Habitat-Sim with Dynamic Gaussian Splatting (GS). It addresses the limitations of traditional mesh-based simulators by enabling photo-realistic rendering and dynamic human avatar simulation within complex 3D environments. This project targets researchers and developers in robotics and AI who require more visually accurate and dynamic simulation platforms for training and evaluating navigation agents. The primary benefit is the ability to conduct embodied AI research in environments that closely resemble real-world visual fidelity and dynamic interactions.

How It Works

Habitat-GS integrates 3D Gaussian Splatting as its rendering backbone, replacing traditional rasterization pipelines. This allows for the rendering of highly detailed and photorealistic scenes directly from GS representations. A key innovation is the incorporation of dynamic Gaussian avatars, powered by models like GaussianAvatar and AnimatableGaussians, whose motion is driven by trajectories generated using the GAMMA method. This approach maintains Habitat-Sim's core functionalities, including NavMesh-based pathfinding, agent control, and integration with the Habitat-Lab framework, while significantly enhancing visual realism and enabling complex agent-environment interactions.

Quick Start & Requirements

• Primary Install: Requires creating a Conda environment (conda create -n habitat-gs python=3.12 cmake=3.27), installing CUDA-compatible PyTorch (pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121), and then installing Habitat-GS itself (git clone ..., cd habitat-gs, git submodule update --init --recursive, HABITAT_WITH_CUDA=ON HABITAT_WITH_BULLET=OFF pip install .).
• Prerequisites: Python 3.12, CMake 3.27, CUDA Toolkit (compatible with PyTorch installation, e.g., 12.1+). Optional dependencies exist for specific features (e.g., GAMMA, SAM, CLIP).
• Data: Significant data downloads are required: GS Scenes (~27 GB), Gaussian Avatars (~3.1 GB), and task-specific datasets (up to ~40 GB for VLN).
• Links:
  • NavMesh Editing Tool: README#navmesh_editor
  • GS Dataset: huggingface.co/datasets/RukawaY/gs_scenes
  • Paper: arxiv.org/abs/2604.12626

Highlighted Details

• Dynamic Gaussian Avatars: Enables simulation of human-like avatars with realistic motion, crucial for human-aware navigation tasks.
• Browser-Based NavMesh Editor: Facilitates intuitive creation and modification of walkable areas directly within GS scenes.
• Multi-Task Support: Provides pipelines for PointNav, ImageNav, ObjectNav, Vision-and-Language Navigation (StreamVLN, Uni-NaVid), and Dynamic Navigation (avoidance, tracking).
• HabitatAgent Integration: Features an LLM-powered agent system for interactive exploration and autonomous navigation via MCP tools.

Maintenance & Community

The project is associated with authors from ZJU and has been accepted to ECCV 2026, indicating active research development. Specific community channels (Discord, Slack) or detailed maintenance information are not explicitly provided in the README.

Licensing & Compatibility

The license type is not specified in the provided README, which represents a significant adoption blocker. Compatibility is designed around the Habitat-Sim ecosystem, but specific CUDA and Python versions are required.

Limitations & Caveats

The setup process involves managing multiple environments and downloading large datasets. Training complex models, particularly for VLN tasks, requires substantial GPU memory (e.g., ≥80 GB VRAM per GPU). The absence of a clear license statement is a critical limitation for assessing commercial use or redistribution rights.