This repository provides the official code for Adaptive-RAG, a framework designed to dynamically select the most suitable retrieval strategy for Large Language Models (LLMs) based on query complexity. It aims to improve the efficiency and accuracy of Question-Answering (QA) systems by balancing single-step, iterative, and no-retrieval methods. The target audience includes researchers and practitioners working on retrieval-augmented LLMs.

How It Works

Adaptive-RAG employs a classifier, a smaller LM trained on automatically collected labels, to predict query complexity. This classifier then dynamically routes queries to the most appropriate retrieval strategy (single-step, iterative, or no-retrieval), optimizing resource usage and response quality across a spectrum of query difficulties. This adaptive approach avoids unnecessary overhead for simple queries and ensures complex queries are handled effectively.

Quick Start & Requirements

• Installation: Create a conda environment (conda create -n adaptiverag python=3.8), activate it, and install dependencies (pip install torch==1.13.1+cu117 --extra-index-url https://download.pytorch.org/whl/cu117, pip install -r requirements.txt).
• Prerequisites: Elasticsearch 7.10.2, Python 3.8, PyTorch with CUDA 11.7. Requires API keys for GPT models.
• Setup: Involves setting up an Elasticsearch retriever server and an LLM server (e.g., for FLAN-T5). Data preparation includes downloading and processing multi-hop and single-hop QA datasets.
• Resources: Requires significant disk space for datasets and indices (e.g., Wiki index ~21M documents).
• Documentation: Official code repository for the NAACL 2024 paper.

Highlighted Details

• Integrates with LlamaIndex, LangChain, and ReAct.
• Supports multiple datasets: MuSiQue, HotpotQA, 2WikiMultiHopQA, Natural Question, TriviaQA, SQuAD.
• Offers pre-trained classifiers for FLAN-T5-XL, FLAN-T5-XXL, and GPT models.
• Evaluates performance on both dev and test sets for various retrieval strategies.

Maintenance & Community

The project is associated with NAACL 2024. Further community or maintenance details are not explicitly provided in the README.

Licensing & Compatibility

The repository does not explicitly state a license. The code is presented as the "Official Code Repository" for a research paper.

Limitations & Caveats

The setup process is complex, requiring the installation and configuration of multiple servers (Elasticsearch, LLM) and extensive data preparation. The reliance on specific versions of PyTorch (1.13.1+cu117) and Elasticsearch (7.10.2) may pose compatibility challenges.