Search-R1 is an open-source reinforcement learning framework for training LLMs to reason and interact with search engines. It targets researchers and developers looking to build tool-augmented LLMs, offering a flexible and extensible alternative to proprietary systems like OpenAI DeepResearch. The framework enables LLMs to learn coordinated search and reasoning strategies, enhancing their ability to answer complex queries.

How It Works

Search-R1 is built upon the veRL framework, extending DeepSeek-R1(-Zero) by integrating interleaved search engine calls. It supports various RL algorithms (PPO, GRPO, reinforce), LLMs (Llama3, Qwen2.5), and search engines (local retrievers, online APIs). This modular design allows for flexible experimentation with different components of the reasoning and search pipeline.

Quick Start & Requirements

• Installation: Requires Python 3.9, PyTorch (cu121 recommended), vLLM (0.6.3 or compatible), flash-attn, and wandb. An optional separate environment for local retrievers requires Python 3.10, PyTorch (cu121), transformers, datasets, pyserini, and faiss-gpu (1.8.0). API function support requires uvicorn and fastapi.
• Setup: Basic setup involves environment creation and package installation. Training requires downloading and indexing corpora, processing datasets (e.g., NQ), launching a retrieval server, and running the training script.
• Links: Paper, Model and data, Twitter thread, Full experiment log, v0.1, v0.2.

Highlighted Details

• Supports multinode training for 30B+ LLMs.
• Integrates local sparse/dense retrievers (with ANN indexing) and online search engines.
• Enables custom datasets and corpora, with examples provided for data processing and corpus formatting.
• Offers flexibility in choosing RL methods, LLMs, and search engine backends.

Maintenance & Community

The project is actively developed with recent updates in April 2025 supporting multinode training and diverse search engines. The paper was published in March 2025. Links to preliminary results and experiment logs are available.

Licensing & Compatibility

The framework is open-source, with a citation provided for the associated arXiv preprint. Specific license details beyond open-source are not explicitly stated in the README.

Limitations & Caveats

The framework is presented with preliminary results, indicating ongoing development. While it supports various components, users may need to configure and integrate specific search engines or datasets according to the provided examples and API specifications.