This repository provides the source code for a foundational guide to Retrieval Augmented Generation (RAG), aimed at beginners in applied generative AI. It offers a practical, code-driven introduction to RAG fundamentals, system design, indexing, generation, and evaluation, enabling users to develop and understand RAG-enabled applications.

How It Works

The project illustrates RAG through a two-pipeline approach: an indexing pipeline that creates a knowledge base by loading, chunking, embedding text, and storing vectors in FAISS, and a generation pipeline that retrieves relevant context, augments user queries, and generates responses using LLMs like GPT-4o. It also explores alternative retrieval algorithms (TF-IDF, BM25) and advanced RAG strategies.

Quick Start & Requirements

• Install: Clone the repository (git clone https://github.com/abhinav-kimothi/A-Simple-Introduction-to-RAG.git), navigate into the directory, create and activate a virtual environment, and run pip install -r requirements.txt.
• Prerequisites: Python version > 3.11.1. Requires API keys for OpenAI and LangChain, which should be stored in a .env file.
• Links: Seminal RAG paper: https://arxiv.org/abs/2005.11401. Book MEAP: https://mng.bz/8wdg.

Highlighted Details

• Code examples demonstrate indexing a Wikipedia article on the 2023 Cricket World Cup using AsyncHtmlLoader, Html2TextTransformer, RecursiveCharacterTextSplitter, OpenAI embeddings, and FAISS.
• Generation pipeline examples utilize FAISS for retrieval and GPT-4o for response generation, with additional notebooks covering TF-IDF and BM25 retrievers.
• Evaluation of RAG pipelines is demonstrated using the RAGAS framework and LangChain Benchmarks.
• Advanced topics include index/query optimization, retrieval strategies, and post-retrieval compression techniques.

Maintenance & Community

The repository is marked as a Work In Progress (WIP), with subsequent chapters and code to be released ongoing. Feedback and contributions are welcomed via the repository's Discussion forum.

Licensing & Compatibility

The README does not explicitly state a software license. This omission requires clarification for assessing commercial use or closed-source integration compatibility.

Limitations & Caveats

The book and repository are still under active development, with content being released incrementally. Some sections, particularly advanced RAG strategies and optimizations, are noted as WIP.