VerseCrafter is a controllable video world model designed to learn realistic and dynamic video priors from large-scale, in-the-wild data. It addresses the challenge of generating videos with strong spatial-temporal coherence while offering explicit, interpretable 4D geometric control over camera and multi-object motion. This project is targeted at researchers and power users who require precise control over video generation for dynamic scenes, aiming to improve geometric faithfulness in generated content.

How It Works

VerseCrafter employs a novel approach by attaching a geometry-aware "GeoAdapter" to a frozen Wan2.1 video prior backbone. This GeoAdapter injects 4D control signals directly into the diffusion blocks, enabling precise control over camera and object trajectories without compromising the quality of the video prior. The system is trained and supervised using the newly introduced VerseControl4D dataset, which features automatically rendered camera trajectories and multi-object 3D Gaussian trajectories derived from real-world videos.

Quick Start & Requirements

Installation involves cloning the repository (including submodules), creating a Conda environment with Python 3.11, and installing specific versions of PyTorch (2.3.0 with CUDA 12.1), along with other dependencies like MoGe, Grounded-SAM-2, and PyTorch3D. A recommended setup includes Blender 4.0+ (4.5+ preferred) and system utilities like ffmpeg and ffprobe. The project offers two usage methods: a Blender Addon (recommended for a one-stop workflow) and a Script Pipeline for offline use. The Blender Addon requires a remote GPU server running an API server (e.g., python api_server.py --port 8188 --num_gpus 8). Detailed instructions are available in README_BLENDER.md.

Highlighted Details

• Accepted to CVPR 2026.
• Provides a unified 4D control state for direct manipulation of camera and multi-object motion.
• Introduces the VerseControl4D dataset for supervising 4D controllable generation.
• Offers both a user-friendly Blender Addon and a flexible Script Pipeline for inference.

Maintenance & Community

The README does not provide specific details regarding notable contributors, sponsorships, or community channels like Discord or Slack.

Licensing & Compatibility

This project is released under the "VerseCrafter License." It is strictly intended for academic/research purposes only, and commercial use is explicitly prohibited.

Limitations & Caveats

Commercial use of VerseCrafter is not permitted due to its restrictive license. The recommended Blender Addon requires network access to a dedicated GPU server, while the script pipeline involves multiple manual steps and integration with Blender for trajectory customization.