Agentic RAG-R1 is an open-source framework designed to enhance the reasoning capabilities of Agentic Retrieval-Augmented Generation (RAG) systems. It targets researchers and developers looking to build more autonomous and effective AI agents by leveraging reinforcement learning for improved decision-making in retrieval and reasoning processes. The primary benefit is a significant boost in both search depth and answer quality for complex query scenarios.

How It Works

The framework employs an agent memory stack to orchestrate a deliberation loop, supporting actions like planning, reasoning, backtracking, summarization, tool observation (e.g., wiki search), and concluding. It utilizes the GRPO (Generalized Relevance Policy Optimization) algorithm, inspired by DeepSeek-R1, to train the agent's choice of reasoning steps and retrieval actions. This approach aims to improve the agent's ability to autonomously decide when and what to retrieve, and how to integrate that information into its responses.

Quick Start & Requirements

• Installation: Uses conda for environment management.

  conda create -n AgenticRAG python=3.11 -y
  conda activate AgenticRAG
  pip install -r requirements.txt
  

• Prerequisites: Requires Python 3.11 and the requirements.txt dependencies. An optional local instance of the ArtSearch tool (for Wikipedia retrieval) is recommended.
• Setup: Requires renaming .env_format to .env and filling in environment variables.
• Resources: Supports models up to 32B parameters on 2 A100 GPUs.
• Links: ArtSearch repository (for search tool deployment).

Highlighted Details

• Reinforcement Learning via GRPO for agentic reasoning and retrieval.
• Supports LoRA tuning, model quantization (nf4), custom agent tools, and distributed training (Deepspeed Zero 2/3).
• Features a tool-calling reward model incorporating accuracy, format, and RAG accuracy (using RAGAS).
• Evaluated on MedQA, showing significant improvements in format and answer accuracy after fine-tuning.

Maintenance & Community

• Contributors include researchers from Peking University.
• Inspired by Deepseek-R1 and TC-RAG.

Licensing & Compatibility

• Licensed under the Apache License.
• Permits commercial use and linking with closed-source applications.

Limitations & Caveats

The project is in an early stage, with "Inference Example Mode" noted as "coming soon." The roadmap indicates planned additions of more tools and features.