This project provides the official PyTorch implementation for "Flexible Motion In-betweening with Diffusion Models," presented at SIGGRAPH 2024. It enables researchers and developers to generate realistic 3D human motion sequences, offering flexible control through text prompts or specified keyframes. The core benefit lies in its diffusion-based approach for high-quality motion synthesis and editing.

How It Works

The system leverages diffusion models to perform motion in-betweening. It supports both unconditional text-to-motion generation and conditional generation, where motion sequences can be guided by user-provided text descriptions or specific spatial keyframes. This dual conditioning capability allows for precise control over motion synthesis, enabling tasks like interpolating between poses or generating novel motions based on semantic input.

Quick Start & Requirements

• Environment: Developed on Ubuntu 20.04 LTS with Python 3.7, CUDA 11.7, and PyTorch 1.13.1.
• Installation: Requires ffmpeg and spacy (with en_core_web_sm model). CLIP is installed via pip install git+https://github.com/openai/CLIP.git.
• Data: Requires downloading SMPL files, GloVe, T2M evaluators, and the HumanML3D dataset. Specific instructions are provided for handling motion representation (absolute root joint).
• Pretrained Models: Download models and place them in the ./save/ directory.
• Usage: Example commands are provided for text-to-motion generation (unconditioned and conditional) and keyframe-conditioned editing.

Highlighted Details

• Official implementation for the SIGGRAPH 2024 paper "Flexible Motion In-betweening with Diffusion Models".
• Enables text-to-motion synthesis.
• Supports keyframe conditioning for motion editing and in-betweening.
• Includes scripts to render generated animations as SMPL meshes.

Maintenance & Community

No specific details regarding community channels (Discord, Slack), active contributors, or roadmap are provided in the README.

Licensing & Compatibility

The code is distributed under an MIT License. However, users must also adhere to the licenses of its dependencies, including CLIP, SMPL, SMPL-X, PyTorch3D, and the HumanML3D dataset. Commercial use compatibility is subject to these underlying licenses.

Limitations & Caveats

The interactive flag for selecting keyframes during generation is noted as being "In development."