ReCall is a framework for training Large Language Models (LLMs) to reason and utilize tools via reinforcement learning, without requiring supervised data on tool use. It aims to enable LLMs to act as general-purpose agents by learning to use and combine arbitrary tools.

How It Works

ReCall employs reinforcement learning to train LLMs to interact with tools. It generates synthetic data for training and supports arbitrary tool integration. The framework is built upon a customized version of the verl reinforcement learning library and utilizes a sandboxed environment for safe execution of tool code.

Quick Start & Requirements

• Installation: Clone the repository and install with pip3 install -e ..
• Prerequisites: Python 3.10, flash-attn, and optionally faiss-gpu for Wikipedia RAG.
• Sandbox: Requires a sandboxed Python execution environment, runnable via python sandbox.py --port {port}.
• Retriever: For Wikipedia RAG, a retriever service using FlashRAG and FastAPI needs to be set up and configured.
• Training: Uses verl (included in src/verl) and requires tool URLs, model paths, and W&B API keys. Example training scripts are provided.
• Inference: Can be served using SGLang.
• Evaluation: Uses FlashRAG for multi-hop QA evaluation.
• Links: Blog, Paper, Models

Highlighted Details

• Trains LLMs for tool use and reasoning via RL without supervised tool-use trajectories.
• Supports arbitrary tool integration and acts as a successor to the ReSearch framework.
• Includes a sandboxed environment for secure tool execution.
• Provides a framework for generating synthetic data for training.

Maintenance & Community

The project is actively being worked on, with the first version released on April 24, 2025. Trained models are available on Hugging Face. The implementation is based on verl, FlashRAG, BFCL, FastAPI, and SGLang.

Licensing & Compatibility

The repository does not explicitly state a license in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project is described as a "work in progress." The current sandbox implementation is basic, with plans for a more robust version. The README notes potential security risks with local sandbox hosting.