<2-3 sentences summarising what the project addresses and solves, the target audience, and the benefit.> LiteReality addresses the challenge of generating high-fidelity, graphics-ready 3D scenes from RGB-D scans. It targets researchers and developers in computer graphics and AR/VR, enabling the creation of scenes with realistic PBR materials directly from captured data. The project was accepted at NeurIPS 2025.

How It Works

The system reconstructs 3D scenes by processing RGB-D scans through distinct object and material painting stages. It leverages large multimodal models like Qwen-VL-8B-Instruct, alongside vision models such as CLIP, DinoV2, and SAM, to infer and apply photorealistic Physically Based Rendering (PBR) materials. The approach aims to automate the creation of visually rich 3D environments suitable for rendering pipelines.

Quick Start & Requirements

• Prerequisites: Linux, Conda, NVIDIA RTX GPU (≥ 24 GB VRAM), CUDA 11.x/12.x.
• Installation: Requires cloning the repository, setting up a Python 3.9 Conda environment, installing PyTorch with CUDA support, and installing the project via pip install -e .. A patched version of GroundingDINO must also be installed.
• Data: A ~200 GB material database and example scans must be downloaded.
• Usage: Test with bash example_scans_test.sh or process custom scans captured via Apple RoomPlan/3D Scanner App.
• Links: The project mentions an arXiv paper and a video demo, but direct URLs are not provided in the README. The official GroundingDINO repository is referenced for potential installation issues.

Highlighted Details

• NeurIPS 2025 accepted paper.
• Generates "graphics-ready" 3D scenes with full PBR material support.
• Outputs include native Blender project files (.blend) and GLB exports.
• Integrates multiple advanced AI models for scene understanding and material application.

Maintenance & Community

The project code was released on January 19, 2026. No specific community channels (e.g., Discord, Slack) or roadmap links are provided in the README.

Licensing & Compatibility

The README does not specify a software license. This omission requires clarification for commercial use or integration into proprietary projects.

Limitations & Caveats

Data capture is currently limited to Apple's RoomPlan framework on LiDAR-equipped iPhones. The setup requires substantial disk space for the material database (~200 GB) and significant GPU VRAM (≥ 24 GB). Installation involves custom patches for third-party dependencies like GroundingDINO.