This project provides an offline, self-contained solution for Retrieval Augmented Generation (RAG), enabling users to ingest local files, GitHub repositories, and websites for querying with open-source LLMs. It targets developers and researchers prioritizing data privacy and avoiding third-party dependencies, offering a fully local RAG pipeline.

How It Works

The system processes various data sources, generates embeddings using local models, and stores them in a vector database. When a query is made, it retrieves relevant information from the vector store and feeds it, along with the query, to a local LLM for a contextually aware response. This approach ensures all data remains within the user's network, enhancing privacy and security.

Quick Start & Requirements

• Install via pip install -r requirements.txt.
• Requires Python 3.10+.
• Recommended: GPU with CUDA 11.8+ for faster embedding generation.
• See Setup & Deploy the App Using Local RAG for detailed instructions.

Highlighted Details

• Offline embedding and LLM support, eliminating reliance on external APIs like OpenAI.
• Ingestion capabilities for local files, GitHub repositories, and websites.
• Features conversational memory and chat export functionality.
• Supports streaming responses for a more interactive user experience.

Maintenance & Community

The project is maintained by jonfairbanks. Community interaction channels are not explicitly mentioned in the README.

Licensing & Compatibility

The project is licensed under the MIT License, permitting commercial use and integration with closed-source applications.

Limitations & Caveats

The project is presented as a personal project, and its long-term maintenance and community support are not yet established. Some features may be experimental or subject to change.