AlayaWorld is an open-source, interactive, long-horizon world model for generating complex video sequences. It tackles temporal and spatial consistency over extended durations while enabling real-time user interaction. This project targets generative AI and simulation researchers/developers, offering a tool for dynamic, controllable virtual environments.

How It Works

Built on interaction, consistency, stability, and runtime, AlayaWorld uses a 3D cache with camera modulation and chunk-level prompt switching for dynamic event introduction. Consistency is maintained via a 3D cache for spatial recall and a frame-history embedding for temporal continuity. Long-horizon stability is achieved through drifted history training and an error bank. Real-time performance comes from DMD distillation and short temporal chunks, minimizing latency.

Quick Start & Requirements

Requires a CUDA GPU and PyTorch >= 2.6. Setup involves downloading AlayaWorld weights (merged_infer.safetensors), the gated Gemma-3 text encoder (google/gemma-3-12b-it-qat-q4_0-unquantized), and Depth-Anything-3 (DA3) code/weights (ByteDance-Seed/Depth-Anything-3, depth-anything/DA3NESTED-GIANT-LARGE-1.1). DA3 must be cloned and installed. Inference runs via inference/run.sh or python -m inference.run with an initial image, camera trajectory, and text prompt.

Highlighted Details

• Long-horizon memory consistency via 3D spatial caches and temporal embeddings.
• Real-time interaction with grounded navigation and mid-generation prompt switching.
• Stability mechanisms: drifted history training and error re-injection.
• Runtime optimization via DMD distillation for minute-long videos.

Maintenance & Community

Led by Kaipeng Zhang, with core contributors from Alaya Lab. Inference code and weights released July 16, 2026. Training code/data planned. Contact: kaipeng.zhang@shanda.com for business.

Licensing & Compatibility

LTX-2 Community License Agreement: academic/research use only. Commercial use restricted, requiring a license from Lightricks Ltd. (especially for entities with >= $10M annual revenue). Original LTX-2 notices retained.

Limitations & Caveats

Training code/data not yet available. Gemma text encoder is gated and requires license acceptance. LTX-2 Community License prohibits commercial use.