MeshGPT generates high-fidelity 3D triangle meshes using a decoder-only transformer architecture. It tokenizes mesh geometry into a learned vocabulary, enabling autoregressive generation of coherent and compact meshes with sharp edges, suitable for researchers and developers in 3D computer vision and graphics.

How It Works

MeshGPT employs a two-stage process: first, a VQ-VAE learns a discrete geometric vocabulary, and second, a transformer model autoregressively samples tokens from this vocabulary. These tokens are then decoded into mesh faces. This approach allows for efficient and high-quality mesh generation by leveraging the power of transformers for sequential data.

Quick Start & Requirements

• Install dependencies: pip install torch-scatter ..., pip install torch==2.1.0 ..., pip install packaging, pip install -r requirements.txt.
• Requires PyTorch 2.1.0 with CUDA 11.8.
• Pretrained models and data must be downloaded and placed in pretrained/ and data/shapenet/ respectively.
• Inference command: python inference/infer_meshgpt.py <ckpt_path> <sampling_mode> <num_samples>.
• Official project page and arXiv paper available.

Highlighted Details

• Generates clean, coherent, and compact meshes with sharp edges and high fidelity.
• Utilizes a learned geometric vocabulary for tokenization.
• Transformer model trained autoregressively on mesh tokens.
• Pretrained models available for ShapeNet chairs and tables.

Maintenance & Community

The project is associated with authors from Audi, Google, and Technical University of Munich. No specific community channels (Discord/Slack) or roadmap are mentioned in the README.

Licensing & Compatibility

The project is licensed under the Automotive Development Public Non-Commercial License Version 1.0. Portions, such as NanoGPT code, are under the MIT license. The non-commercial clause restricts use in commercial products.

Limitations & Caveats

The primary license restricts commercial use. The README does not detail specific hardware requirements beyond CUDA for PyTorch, nor does it provide benchmarks against other mesh generation methods.