GaussianObject is a framework for high-quality 3D object reconstruction using Gaussian Splatting, specifically designed for scenarios with only four input views, even without precise camera poses (COLMAP-free). It targets researchers and practitioners in computer vision and graphics seeking efficient and detailed 3D object modeling from limited imagery.

How It Works

GaussianObject employs a two-stage approach. First, it generates a coarse 3D Gaussian representation by incorporating visual hull and floater elimination techniques to enforce multi-view consistency. Second, a diffusion model-based Gaussian repair mechanism refines this representation, filling in omitted details and improving fidelity. This hybrid strategy leverages structural priors and generative models for robust reconstruction from minimal data.

Quick Start & Requirements

• Installation: Clone the repository with --recursive and install dependencies via pip install -r requirements.txt. PyTorch with CUDA 11.8 is recommended.
• Prerequisites: CUDA 11.8, Python 3.11. For COLMAP-free operation, checkpoints for SAM and DUSt3R (or MASt3R) are required.
• Data: Mip-NeRF360 or OmniObject3D datasets are supported, with custom data requiring a specific directory structure.
• Resources: Requires significant GPU VRAM, with a Colab option available for lower-resource environments.
• Links: Project Page, Paper, Video

Highlighted Details

• Achieves high-quality 3D reconstruction from as few as 4 input views.
• Offers a COLMAP-free pipeline using SAM for masks and DUSt3R/MASt3R for pose estimation.
• Integrates diffusion models for Gaussian repair and refinement.
• Includes a "leave-one-out" strategy for training the repair model.

Maintenance & Community

The project is associated with SIGGRAPH Asia 2024 and ACM Transactions on Graphics, indicating strong academic backing. Code is based on 3DGS, threestudio, and ControlNet.

Licensing & Compatibility

The repository does not explicitly state a license in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project is presented as a research artifact, and extensive testing on diverse datasets or real-world applications may be limited. The setup for COLMAP-free reconstruction requires downloading and integrating additional model checkpoints.