RAG method for improved generation robustness
Top 73.7% on sourcepulse
This repository provides the source code for Corrective Retrieval Augmented Generation (CRAG), a method to enhance the robustness of Large Language Models (LLMs) against inaccurate or irrelevant retrieved documents in Retrieval-Augmented Generation (RAG) systems. It is designed for researchers and developers working with LLMs and RAG to improve generation quality and reduce hallucinations.
How It Works
CRAG introduces a lightweight retrieval evaluator that assesses the quality of retrieved documents and assigns a confidence score. This score dynamically triggers different knowledge retrieval actions, including internal document decomposition and external web searches. A decompose-then-recompose algorithm further refines retrieved documents by focusing on key information and filtering out irrelevant content, making CRAG a plug-and-play module for existing RAG approaches.
Quick Start & Requirements
pip install -r requirements.txt
.Highlighted Details
Maintenance & Community
The project was last updated on October 8, 2024, with revisions to prompt formatting. Key releases include inference code, evaluator weights, and data preparation scripts. Further details on community channels or active development are not explicitly mentioned in the README.
Licensing & Compatibility
The repository does not explicitly state a license. The code is provided for research purposes, and citation of the associated paper is requested. Commercial use or linking with closed-source projects may require further clarification on licensing terms.
Limitations & Caveats
The evaluation results reported in the paper were based on the deprecated text-davinci-003
model, and may differ when using current models like gpt-3.5-turbo-instruct
. The README also notes that the Self-CRAG inference and evaluator fine-tuning were planned for future release at the time of writing.
9 months ago
Inactive