OmniNWM is a unified panoramic navigation world model designed to advance autonomous driving simulation. It addresses the need for joint generation of multi-modal states (RGB, semantics, depth, 3D occupancy), precise action control using normalized Plücker ray-maps, and facilitates closed-loop evaluation through occupancy-based dense rewards. This project benefits researchers and engineers in autonomous driving by providing a comprehensive simulation environment.

How It Works

The core approach involves jointly generating panoramic multi-modal states, including RGB, semantic, metric depth, and 3D occupancy videos. Action control is achieved via normalized Plücker ray-maps, enabling pixel-level trajectory interpretation. The system facilitates closed-loop evaluation by integrating occupancy-based dense rewards, promoting realistic driving policy assessment and ensuring driving compliance and safety. A flexible forcing strategy allows for auto-regressive generation beyond ground truth length, enhancing long-term stability.

Quick Start & Requirements

The README mentions a demo released on the Project Page as of September 2025. However, specific installation commands, non-default prerequisites (like GPU, CUDA, Python versions), or estimated setup times are not detailed in the provided text. Links to official quick-start guides or documentation are also absent.

Highlighted Details

• Multi-modal Generation: Jointly generates RGB, semantic, depth, and 3D occupancy in panoramic views.
• Precise Camera Control: Utilizes normalized Plücker ray-maps for pixel-level trajectory interpretation.
• Long-term Stability: Employs a flexible forcing strategy for auto-regressive generation beyond ground truth length.
• Closed-loop Evaluation: Integrates occupancy-based dense rewards for realistic driving policy evaluation.
• Zero-shot Generalization: Demonstrates transferability across datasets and camera configurations without fine-tuning.

Maintenance & Community

The project acknowledges being built upon OpenSora and Qwen-VL. No specific details regarding core maintainers, community channels (like Discord/Slack), roadmaps, or sponsorships are provided in the README.

Licensing & Compatibility

The project is licensed under the Apache License 2.0. This license is generally permissive and compatible with commercial use and closed-source linking, allowing for broad adoption.

Limitations & Caveats

The provided README does not explicitly detail limitations, known bugs, or alpha/beta status. The recent release of a demo (September 2025) suggests the project may still be under active development. Specific hardware or software requirements beyond general autonomous driving simulation needs are not detailed.