LLaMA-Mesh enables the creation of 3D meshes through conversational interaction with large language models (LLMs). It targets researchers and developers interested in unifying 3D generation with natural language processing, allowing for text-based 3D mesh creation and understanding. The primary benefit is leveraging LLMs' existing spatial knowledge from text and enabling intuitive, chat-driven 3D content generation.

How It Works

LLaMA-Mesh tokenizes 3D mesh data (vertex coordinates and face definitions) into plain text, allowing direct integration with LLMs without vocabulary expansion. This approach facilitates end-to-end training on interleaved text and 3D mesh data, enabling a single model to generate both text and 3D meshes. This unification allows LLMs to acquire complex spatial knowledge for 3D mesh generation directly from textual sources.

Quick Start & Requirements

• Install/Run: Use Gradio UI (python app.py) or integrate with Hugging Face Transformers:

  from transformers import AutoModelForCausalLM, AutoTokenizer
  model_path = "Zhengyi/LLaMA-Mesh"
  tokenizer = AutoTokenizer.from_pretrained(model_path)
  model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto")
  

• Prerequisites: Requires Python and Hugging Face Transformers. GPU acceleration is recommended for inference.
• Resources: Model weights are available on Hugging Face.
• Links: Project Page, Huggingface Demo, Blender Addon (Note: Link appears to be a direct asset link, not a repo).

Highlighted Details

• Achieves mesh generation quality comparable to models trained from scratch.
• Maintains strong text generation performance alongside 3D capabilities.
• Supports interleaved text and 3D mesh outputs.
• Enables understanding and interpretation of 3D meshes via LLMs.

Maintenance & Community

The project acknowledges Dylan Ebert for the Blender addon implementation. Further community and roadmap details are not explicitly provided in the README.

Licensing & Compatibility

The README does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The training dataset is not yet uploaded. The project is presented as a research artifact, and extensive testing or production readiness is not detailed.