This project provides a fast and generalizable single-view 3D reconstruction system, targeting researchers and developers in computer vision and graphics. It enables high-quality 3D reconstruction from a single image in seconds, leveraging a novel hybrid Triplane-Gaussian representation.

How It Works

The system employs a hybrid 3D representation combining Triplane and Gaussian Splatting. Triplanes offer an efficient implicit representation, while Gaussian Splatting provides explicit, high-fidelity rendering. This fusion allows for fast inference and high-quality results by capturing both global structure and fine details. Transformers are utilized to process the input image and guide the reconstruction process.

Quick Start & Requirements

• Installation: pip install -r requirements.txt (after installing PyTorch, pointnet2_ops, pytorch_scatter, and diff-gaussian-rasterization).
• Prerequisites: Python >= 3.8, PyTorch >= 1.12 (tested with cu113), CUDA 11.3, pointnet2_ops, pytorch_scatter, diff-gaussian-rasterization, PyTorch3D.
• Pretrained Model: Download from Hugging Face (VAST-AI/TriplaneGaussian).
• Demo: Online Gradio demo available on Hugging Face Spaces. Colab notebook provided.
• Links: Hugging Face Demo, Colab Demo

Highlighted Details

• Achieves high-quality 3D reconstruction from single-view images in under a second.
• Utilizes a novel hybrid Triplane-Gaussian 3D representation.
• Compatible with graphdeco-inria/gaussian-splatting PLY format.
• Supports background removal via SAM checkpoint integration.

Maintenance & Community

• Official implementation of the paper "Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers".
• Supported by Tsinghua University and VAST.
• Code modified from SnowflakeNet for point cloud upsampling.

Licensing & Compatibility

• The repository does not explicitly state a license in the README.

Limitations & Caveats

• The provided pretrained model is trained only on the Objaverse-LVIS dataset.
• Performance may improve with models trained on larger datasets or with more parameters.
• Results can be sensitive to the cam_dist parameter, requiring tuning for optimal output.