PhysX-Omni addresses the challenge of generating unified, simulation-ready 3D assets for rigid, deformable, and articulated objects. It targets researchers and engineers in robotics, simulation, and 3D content creation, offering a framework to synthesize physically plausible 3D models from various inputs.

How It Works

The project integrates large language models (Qwen2.5) with advanced rendering and geometry processing techniques (TRELLIS, nvdiffrast, xformers, flash-attn) to achieve unified 3D generation. It employs a pipeline involving dataset preprocessing, conditioning image rendering, model finetuning, and inference for generating geometry, which can then be converted into simulation-ready formats like URDF and XML. This approach aims for high-fidelity and physically accurate 3D asset synthesis.

Quick Start & Requirements

Installation involves cloning the repository with submodules and running a setup script (./setup.sh --new-env --basic --xformers --flash-attn --diffoctreerast --spconv --mipgaussian --kaolin --nvdiffrast) to create a conda environment and install core dependencies, or alternatively, using a provided requirements.txt with Python 3.10. Key prerequisites include downloading large datasets from Hugging Face (PhysXNet, PhysX-Mobility, PhysXVerse) and potentially a pre-trained TRELLIS decoder for enhanced geometric detail. Setup requires careful environment management and data preparation. Links to TRELLIS, datasets, and pre-trained models are provided.

Highlighted Details

• Unified generation capabilities for rigid, deformable, and articulated objects.
• Leverages multimodal large language models (Qwen2.5) for generative tasks.
• Includes PhysX-Bench for performance evaluation and PhysXVerse dataset.
• Provides tools for converting generated assets into simulation-ready URDF/XML formats.

Maintenance & Community

The README acknowledges contributions from several foundational open-source projects (Qwen, TRELLIS, etc.) but does not detail specific community channels (like Discord/Slack), active maintainers, or a public roadmap.

Licensing & Compatibility

The project is distributed under the "S-Lab License". The specific terms, restrictions, and compatibility for commercial use or integration with closed-source systems are not detailed in the README and require further investigation.

Limitations & Caveats

The README does not explicitly state limitations, alpha status, or known bugs. The primary caveat is the undefined "S-Lab License," which necessitates a thorough review before adoption, especially for commercial applications. The setup process is complex, involving multiple dependencies and large dataset downloads.