GaussianDreamer is a framework for fast text-to-3D Gaussian splatting generation, targeting researchers and developers in 3D content creation. It bridges the strengths of 2D and 3D diffusion models to produce high-quality, real-time renderable 3D assets from text prompts, significantly reducing generation time compared to prior methods.

How It Works

GaussianDreamer leverages a hybrid approach, using a 3D diffusion model for initial priors and a 2D diffusion model for refining geometry and appearance. It introduces novel operations like noisy point growing and color perturbation to enhance the initialized Gaussian representations, achieving a balance between 3D consistency and high-fidelity generation.

Quick Start & Requirements

• Installation: Requires PyTorch with CUDA 11.7, Ninja, and cloning the repository. Dependencies include shap-e and specific submodules for Gaussian rasterization and kNN.
• Prerequisites: Python 3.x, GPU with CUDA 11.7. A finetuned Shap-E model is required.
• Demo: Huggingface and Colab demos are available.
• Docs: Project page and arXiv paper provide detailed information.

Highlighted Details

• Generates high-quality 3D instances or avatars within 15 minutes on a single GPU.
• Achieves competitive results on ViT similarity and T3 Bench benchmarks.
• Supports integration with Unity game engine via UnityGaussianSplatting.
• Offers extensions for threestudio.

Maintenance & Community

The project is associated with CVPR 2024 and has seen recent updates, including the release of GaussianDreamerPro. It acknowledges contributions from other open-source projects.

Licensing & Compatibility

The repository's licensing is not explicitly stated in the README, but it mentions borrowing code from other projects, implying potential licensing considerations for commercial use.

Limitations & Caveats

The initial code release may contain issues, as noted by the authors. The specific licensing for commercial use is not detailed, which could be a factor for adoption.