This repository provides a comprehensive, modular codebase for building and understanding Retrieval-Augmented Generation (RAG) systems, targeting engineers and researchers focused on practical, business-oriented RAG implementations. It offers a structured approach to mastering the entire RAG pipeline through code examples and practical projects.

How It Works

The project breaks down RAG into ten distinct modules, covering data loading, chunking, embedding, vector storage, retrieval optimization, indexing, response generation, and evaluation. It supports both LangChain and LlamaIndex frameworks, allowing users to explore different implementations and choose the best fit for their needs. The modular design facilitates a deep dive into each component's intricacies and optimization strategies.

Quick Start & Requirements

• Installation: Requires Python 3.10+ and installation via pip install -r <requirements_file>.txt. Specific requirements files are provided for LangChain/LlamaIndex, GPU/CPU, and OS (Ubuntu, macOS, Windows via WSL2).
• Hardware: NVIDIA GPU with >= 8GB VRAM and CUDA 11.8+ recommended for GPU versions. 16GB RAM and a multi-core CPU are recommended for CPU versions.
• Dependencies: Includes libraries like HuggingFace, BGE, Milvus, Chroma, LangChain, LlamaIndex, pandas, PyPDF2, RAGAS, and TruLens. PDF processing may require system-level installs like ghostscript.
• Setup: Environment setup involves creating a virtual environment and installing dependencies. Detailed instructions are provided for various OS and framework combinations.

Highlighted Details

• Supports both LangChain and LlamaIndex frameworks.
• Covers advanced RAG techniques like Graph RAG and Multi-Agent systems.
• Includes system evaluation modules using RAGAS and TruLens.
• Accompanied by a published book, "RAG实战课".

Maintenance & Community

The project welcomes contributions via Issues and Pull Requests. Links to community channels or roadmaps are not explicitly provided in the README.

Licensing & Compatibility

The project is licensed under the MIT License, permitting commercial use and integration with closed-source projects.

Limitations & Caveats

Specific requirements for PDF processing (e.g., ghostscript, python3-tk) may necessitate additional system-level installations beyond Python package management. Some modules might require downloading additional models or configuring API keys.