This repository provides a PyTorch Lightning implementation of 3D Gaussian Splatting, offering a flexible framework for researchers and developers to train and experiment with various derived algorithms. It supports multiple dataset formats, includes an interactive web viewer for scene editing and rendering, and boasts a wide array of advanced features like deformable and dynamic object handling, and appearance modeling.

How It Works

The core of the framework leverages PyTorch Lightning for structured training, abstracting away much of the boilerplate code. It integrates with different rasterization backends, including diff-gaussian-rasterization and nerfstudio-project/gsplat, allowing users to switch between them. The implementation supports advanced techniques such as appearance embeddings for handling varying image conditions and depth regularization for improved scene reconstruction quality.

Quick Start & Requirements

• Installation: Clone the repository, create a conda environment, and install PyTorch matching your CUDA version (e.g., pip install -r requirements/pyt201_cu118.txt), followed by other requirements (pip install -r requirements.txt).
• Prerequisites: Python 3.9+, PyTorch (version-dependent on CUDA), ffmpeg for video rendering. Optional packages for specific features like gsplat, SAM, and pytorch3d are available via separate requirements files.
• Training: Basic training command: python main.py fit --data.path DATASET_PATH -n EXPERIMENT_NAME.
• Web Viewer: Run with python viewer.py TRAINING_OUTPUT_PATH.
• Documentation: Extensive usage instructions and configuration options are detailed within the README.

Highlighted Details

• Supports a wide range of derived algorithms including Deformable Gaussians, Mip-Splatting, LightGaussian, AbsGS/EfficientGS, 2D Gaussian Splatting, and Segment Any 3D Gaussians.
• Features an interactive web viewer for scene transformation, camera path editing, and video rendering.
• Implements advanced training strategies for handling large datasets without OOM errors and offers multi-GPU/node training capabilities.
• Includes novel features like appearance models for varied image conditions and depth regularization using Depth Anything V2.

Maintenance & Community

The project appears actively maintained with frequent updates and additions of new features and algorithms. Links to community resources like Discord/Slack are not explicitly provided in the README.

Licensing & Compatibility

The repository's licensing is not explicitly stated in the provided README snippet. Compatibility for commercial use or closed-source linking would require clarification of the license.

Limitations & Caveats

The README notes that multi-GPU training with the new strategy is not well-validated and is still under development. Some derived algorithms may have specific dependencies or limitations (e.g., gsplat requiring a modified v1).