Summary

PhysX-Anything addresses the challenge of creating simulation-ready 3D assets from single images. It targets engineers and researchers by automating the generation of physically grounded 3D models, streamlining workflows for robotics, simulation, and virtual environments. The primary benefit is enabling rapid asset creation with inherent physical properties derived from minimal input.

How It Works

The system employs a multi-stage pipeline. It begins with Vision-Language Model (VLM) inference, leveraging Qwen2.5, to interpret the input image. Subsequent stages involve a decoder for generating asset geometry, mesh splitting to delineate distinct parts, and a final conversion process to output URDF and XML files. This approach aims to capture not just visual appearance but also physical characteristics crucial for simulation.

Quick Start & Requirements

Installation requires cloning the repository and running a setup script (./setup.sh) which creates a conda environment and installs dependencies including xformers, flash-attn, spconv, kaolin, and nvdiffrast. Additional Python packages like transformers==4.50.0 and qwen-vl-utils are needed. Pre-trained models must be downloaded separately from Hugging Face. Inference involves running a sequence of Python scripts for VLM, decoder, mesh splitting, and URDF/XML generation. Detailed setup guidance can be found via the TRELLIS project.

Highlighted Details

• Generates "Simulation-Ready Physical 3D Assets" directly from a single input image.
• Integrates Qwen2.5 for advanced VLM capabilities in asset interpretation.
• Outputs standard simulation formats: URDF and XML.
• Introduces the accompanying PhysX-Mobility dataset for further research.

Maintenance & Community

The project originates from researchers at Nanyang Technological University and Shanghai AI Laboratory. Specific community channels (e.g., Discord, Slack) or active maintenance signals are not detailed in the README.

Licensing & Compatibility

The project is distributed under the "S-Lab License." The specific terms, restrictions, and compatibility for commercial use or closed-source linking are not elaborated upon in the provided README.

Limitations & Caveats

The README notes that while the system can generate parts with deformable parameters, these components are currently unstable in the MuJoCo simulator. For more reliable simulation outcomes, users are advised to disable the deformable flag. The precise implications of the S-Lab License are not specified.