Verl-Tool is a framework for training tool-using agents, built upon the verl RL library. It enables large language models to interact with external tools through a unified API, facilitating complex task execution and providing a flexible "tool-as-environment" paradigm for reinforcement learning.

How It Works

Verl-Tool decouples RL training from tool interaction by integrating a tool server. The core modification is in verl's ActorRolloutRefWorker, which is extended to delegate agent rollout to an AgentActorManager. This manager orchestrates multi-turn interactions between the LLM and the tool server, allowing the LLM to call tools and receive observations. This design simplifies adding new tools and supports asynchronous rollout generation for improved efficiency.

Quick Start & Requirements

• Installation: Recommended via uv.

  git submodule update --init --recursive
  uv sync
  source .venv/bin/activate
  uv pip install -e verl
  uv pip install -e ".[vllm,acecoder,torl,search_tool]"
  uv pip install "flash-attn<2.8.0" --no-build-isolation
  

• Prerequisites: Python 3.10+, uv, git, vLLM, flash-attn. GPU acceleration is highly recommended for training and inference.
• Setup: Requires cloning the repository and initializing submodules.

Highlighted Details

• Supports trajectory-level asynchronous rollout, doubling rollout generation speed.
• Features a "tool-as-environment" paradigm where tool interactions modify environment state.
• Offers a user-friendly evaluation suite that wraps interactions into an OpenAI-like API.
• Achieves strong performance on mathematical benchmarks, outperforming baseline models with tool integration.

Maintenance & Community

• Actively updated with new features like NL2SQL and DAPO recipe training.
• Provides links to Discord and DeepWiki for documentation and community interaction.
• Core contributors and advisors are listed.

Licensing & Compatibility

• The repository does not explicitly state a license in the README. The underlying verl library is Apache 2.0 licensed. Compatibility for commercial use or closed-source linking would require clarification on the specific license of verl-tool.

Limitations & Caveats

• Multi-node training is experimental.
• Some configuration parameters may require adjustments when updating the verl submodule.
• The specific license for verl-tool itself is not clearly stated, which could impact commercial adoption.