Summary

This repository serves as a comprehensive survey of the rapidly advancing field of 2D human motion video generation. It meticulously curates research papers, datasets, and code, categorizing advancements by driving modality (Vision, Text, Audio) and incorporating LLM-based motion planning. The project aims to provide a structured, up-to-date knowledge base for researchers and practitioners, accelerating progress in creating realistic and controllable human motion videos.

How It Works

The project organizes human motion video generation research into a five-stage pipeline: Input (defining driving source and region), Motion Planning (using feature mapping or Large Language Models), Motion Video Generation (often employing Diffusion Models or Transformers), Video Refinement (focusing on specific body parts like faces or hands), and Acceleration (optimizing for real-time performance). This framework offers a novel and detailed perspective on the field, analyzing both motion planning and generation aspects.

Quick Start & Requirements

This repository functions as a curated knowledge base and does not offer direct installation or execution instructions for a specific tool. It lists research papers and their associated links, serving as a guide to existing work. The authors announce an upcoming benchmark release that will include enhanced evaluation metrics, comprehensive datasets, and fast implementations.

Highlighted Details

• Presents a novel five-stage framework for human motion video generation, covering diverse driving sources and body regions.
• Provides an in-depth analysis of motion planning and motion generation techniques, a dimension often overlooked in prior surveys.
• Clearly outlines established baselines and evaluation metrics, offering critical insights into current challenges.
• Identifies promising future research directions to guide the community's efforts.

Maintenance & Community

The project is actively maintained by a core team of six researchers from leading institutions, with guidance from several academic and industry experts. Contributions are actively encouraged via pull requests, fostering a collaborative environment for advancing the field. For inquiries, contact WinniyGD@outlook.com.

Licensing & Compatibility

The project is released under the MIT License, allowing for broad use and adaptation of the curated information.

Limitations & Caveats

The survey explicitly excludes research involving 3D Generative Scene (3DGS) and Neural Radiance Fields (NeRF) technologies that utilize 2D-3D-2D pipelines. The release of the announced benchmark is subject to the authors' current academic commitments.