Summary

VeRL-Omni provides an easy, fast, and stable Reinforcement Learning (RL) training framework specifically designed for diffusion and omni-modality generative models. It targets researchers and engineers, enabling accelerated development and deployment of advanced multimodal AI systems by addressing unique training challenges beyond traditional LLM RL.

How It Works

Built upon the verl foundation, VeRL-Omni tackles the distinct I/O, compute, and runtime bottlenecks of multimodal generative RL. Its core approach leverages specialized rollout via vLLM-Omni for high-throughput inference, flexible reward pipelines (rule-based, model-based, multimodal), and modular training backends that integrate seamlessly with existing parallelism frameworks like FSDP and USP. This design prioritizes efficiency and adaptability for complex generative tasks.

Quick Start & Requirements

Installation and detailed usage instructions are available in the official documentation. Docs. Specific hardware configurations, such as Ascend NPUs, may require dedicated setup guides.

Highlighted Details

• Achieves ~25% higher end-to-end training throughput compared to diffusers-based flow_grpo implementations on reference setups like Qwen-Image FlowGRPO, attributed to vLLM-Omni rollout and asynchronous reward computation.
• Supports RL post-training for Diffusion models (image, video, audio), Unified multimodal understanding + generation models (text+image), and Omni-modality models (text, image, audio, video).
• Includes Ascend NPU support for specialized hardware acceleration.
• Offers flexible reward pipeline implementations and integrates with standard parallelism strategies (FSDP, USP).

Maintenance & Community

The project is actively developed, with contributions welcomed via its contribution guide. Key contributors include Yongxiang Huang, Cheung Kawai, Jingan Zhou, Yingshan Chen, and the {openYuanrong Team}. Future development plans are tracked in the RFC: Multi-modal Generation RL 2026Q2 Roadmap.

Licensing & Compatibility

VeRL-Omni is released under the Apache 2.0 license. This permissive license allows for broad compatibility, including commercial use and integration within closed-source projects.

Limitations & Caveats

Several model and algorithm combinations are marked as Work In Progress (WIP) or Planned, indicating ongoing development and potential instability for those specific features. The project's roadmap extends into Q2 2026, suggesting it is a relatively new or rapidly evolving framework.