FlowMDM addresses the challenge of generating long, continuous human motion sequences guided by varying textual descriptions, a common limitation in existing models. It targets researchers and developers in VR, gaming, and robotics, offering seamless motion composition without post-processing.

How It Works

FlowMDM is a diffusion-based model that introduces Blended Positional Encodings (BPE) to achieve seamless motion composition. BPE combines absolute and relative positional encodings within the denoising process: absolute encodings restore global motion coherence, while relative encodings ensure smooth, realistic transitions between motion segments. This approach, coupled with Pose-Centric Cross-Attention, enables robust generation from single text descriptions.

Quick Start & Requirements

• Install: Create and activate a conda environment using environment.yml, then run python -m spacy download en_core_web_sm, pip install git+https://github.com/openai/CLIP.git, and pip install git+https://github.com/GuyTevet/smplx.git.
• Prerequisites: Ubuntu 20.04.6 LTS, Python 3.8, PyTorch 1.13.0, ffmpeg, spaCy, CLIP, and smplx.
• Setup: Requires setting up a specific conda environment. Detailed instructions for visualization, evaluation, and training are available in the runners README.

Highlighted Details

• Official implementation for the CVPR 2024 paper "Seamless Human Motion Composition with Blended Positional Encodings."
• Achieves state-of-the-art results on Babel and HumanML3D datasets for accuracy, realism, and smoothness.
• Proposes two new metrics, Peak Jerk and Area Under the Jerk, for evaluating motion transition abruptness.
• Leverages Pose-Centric Cross-Attention for robustness against varying text descriptions.

Maintenance & Community

The project is associated with the CVPR 2024 publication. Key code contributions are noted in model/FlowMDM.py, diffusion/diffusion_wrappers.py, and model/x_transformers. The project inherits significant code from TEMOS, TEACH, MDM, PriorMDM, and x-transformers.

Licensing & Compatibility

The repository does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project is marked as "official implementation" and has released code and model weights, indicating a stable state. However, a "TODO List" indicates that demo-style visualization code is still pending release.