MemoRAG is a Retrieval-Augmented Generation (RAG) framework designed to enhance information retrieval and response generation by leveraging a super-long memory model. It targets applications requiring a global understanding of extensive datasets, offering more accurate and contextually rich outputs than standard RAG.

How It Works

MemoRAG utilizes a memory model to achieve a global understanding of an entire database, going beyond explicit information needs. By recalling query-specific clues from this memory, it improves evidence retrieval. This approach allows for handling up to 1 million tokens in a single context, with features like efficient caching (up to 30x speedup) and context reuse.

Quick Start & Requirements

• Installation: pip install memorag or install from source. GPU with CUDA is recommended.
• Dependencies: torch, faiss-gpu.
• Demo: Available via Google Colab: https://colab.research.google.com/drive/1fPMXKyi4AwWSBkC7Xr5vBdpPpx9gDeFX?usp=sharing
• Examples: Notebooks for Lite Mode, basic usage, and long LLMs as memory models are provided.

Highlighted Details

• Supports up to 1 million tokens context window.
• Achieves up to 30x faster context pre-filling through efficient caching.
• Can be fine-tuned for new tasks with a few hours of training.
• Offers a "Lite Mode" for simplified usage with minimal code.

Maintenance & Community

The project is under active development, with recent updates including support for Llama 3.1 and Qwen2 as memory models. Training scripts and datasets were released in April 2025. Roadmap includes speed improvements and broader retrieval method integration.

Licensing & Compatibility

Licensed under the Apache 2.0 License, permitting commercial use and integration with closed-source applications.

Limitations & Caveats

While MemoRAG supports millions of tokens, performance may degrade for languages other than English if default prompts are used. The roadmap indicates ongoing work to speed up inference and integrate more retrieval methods.