DeepMesh provides an auto-regressive transformer-based approach for generating high-quality 3D meshes conditioned on input point clouds. It is designed for researchers and artists interested in automated 3D content creation, offering a novel method for mesh generation that leverages reinforcement learning.

How It Works

DeepMesh employs an auto-regressive transformer architecture to generate meshes. It iteratively predicts mesh vertices and faces, conditioned on an input point cloud. This approach allows for fine-grained control and high-fidelity mesh generation, with the transformer's attention mechanism enabling it to capture complex geometric relationships. The use of reinforcement learning is intended to optimize the generation process for quality and adherence to the input shape.

Quick Start & Requirements

• Installation: Clone the repository and create a conda environment using environment.yaml. Alternatively, manual installation for CUDA 12.1 is provided, requiring PyTorch 2.5.1 and specific builds of xformers and flash-attention.
• Pretrained Weights: Download weights from Hugging Face using huggingface-cli login and huggingface-cli download zzzrw/DeepMesh --local-dir ./.
• Prerequisites: Ubuntu 22, CUDA 11.8 or 12.1, Python 3.12. Requires significant GPU memory (A100, A800, A6000 mentioned).
• Usage: Inference is performed via command-line using torchrun with specified model paths, input/output directories, and generation parameters.
• More Info: Project Page

Highlighted Details

• Official code for "DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learning".
• Optimized inference code offers a 50% reduction in generation time.
• Pretrained weights for a 0.5B parameter model are available.
• Supports generation from point clouds with or without normals.

Maintenance & Community

The project is associated with Tsinghua University and ShengShu. A "Todo" list indicates plans for releasing pre-training and post-training code, as well as a larger 1B parameter model.

Licensing & Compatibility

The repository does not explicitly state a license in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project is still under active development, with plans for larger models and additional training code. The README does not specify the license, which could impact commercial adoption.