Summary

FlashDreams is a high-performance inference and serving library for interactive autoregressive video and world models. Evolving from NVIDIA's OmniDreams demo, it provides an optimized runtime for real-time applications in gaming, autonomous vehicles, robotics, and simulated environments, enabling efficient deployment of advanced world-model functionalities.

How It Works

This library offers an optimized runtime for interactive autoregressive video and world models, derived from NVIDIA's OmniDreams closed-loop demo. Its core advantage is high-performance inference and serving, enabling real-time interaction and deployment of complex world-model applications, prioritizing efficiency for demanding simulated and virtual environments.

Quick Start & Requirements

• Installation: Clone the repo, sync with uv sync --extra runners, set HF_TOKEN, and run via uv run flashdreams-run. Alternatively, pip install flashdreams.
• Prerequisites: NVIDIA GPU (80 GB VRAM+), NVIDIA driver R580+ (CUDA 13.x), CUDA 13.x, Python >= 3.10, PyTorch >= 2.11.0+cu130, Linux x86-64/arm64, 100 GB+ storage. Docker with NVIDIA Container Toolkit is optional.
• Resources: Development targets 80 GB+ VRAM GPUs; inference may fail on lower-spec hardware.
• Documentation: Links to detailed installation, quickstart, interactive demo, and model gallery guides are mentioned within the README but not provided as direct URLs.

Highlighted Details

• Supports models: Self-Forcing, OmniDreams, LingBot-World, Wan2.1, Causal-Forcing, Causal Wan2.2, FlashVSR, Cosmos-Predict2.5.
• Enables real-time interaction for autonomous vehicle simulation and I2V models.
• Features an interactive driving demo.

Maintenance & Community

• Contributions welcomed via GitHub Issues for integrations, bugs, features, performance, and docs.
• Community support and discussion on NVIDIA Omniverse Discord (#flashdreams channel).
• Developer workflow: uv sync --extra dev --extra runners, uv run pre-commit run -a.

Licensing & Compatibility

• Apache License 2.0.
• REUSE-compliant; third-party licenses in THIRD-PARTY-NOTICES/NOTICE. Permissive for commercial use.

Limitations & Caveats

• Strict hardware requirements (80 GB VRAM+ NVIDIA GPU) are a significant barrier. Inference is prone to out-of-memory errors on less capable GPUs.