IncarnaMind allows users to query personal documents (PDF, TXT) using various LLMs, including OpenAI's GPT series, Anthropic's Claude, and local open-source models like Llama2. It addresses limitations in traditional RAG by offering adaptive chunking and multi-document querying, enabling more precise and context-aware information retrieval.

How It Works

IncarnaMind employs a "Sliding Window Chunking" mechanism for adaptive data segmentation, balancing fine-grained and coarse-grained information access. This is coupled with an "Ensemble Retriever" to enhance both semantic understanding and precise retrieval across multiple documents. This approach aims to overcome the limitations of fixed chunking and single-document querying found in many RAG systems.

Quick Start & Requirements

• Installation: Clone the repository, create a Conda environment (Python 3.8-3.10), install requirements (pip install -r requirements.txt), and optionally llama-cpp-python with CUDA or Metal support.
• Prerequisites: API keys for OpenAI, Anthropic, Together.ai, or a HuggingFace token for local models.
• Setup: Configure API keys and optional parameters in configparser.ini. Place documents in the /data directory.
• Usage: Ingest documents with python docs2db.py, then start chatting with python main.py.
• Resources: Running quantized Llama2-70b-gguf requires >35GB GPU RAM.
• Docs: Demo

Highlighted Details

• Supports multiple LLMs: GPT-3.5, GPT-4 Turbo, Claude 2.0, Llama2 (full and GGUF).
• Adaptive Sliding Window Chunking for improved RAG performance.
• Multi-document conversational QA capabilities.
• Tested Llama2-70b-chat (GGUF) requires significant GPU RAM (>35GB).

Maintenance & Community

• Project is actively maintained by Junru Xiong.
• Acknowledgements mention contributions from Langchain, Chroma DB, LocalGPT, and Llama-cpp.
• Citation details are provided.

Licensing & Compatibility

• Licensed under the Apache 2.0 License.
• Permissive license suitable for commercial use and integration into closed-source projects.

Limitations & Caveats

Citation functionality is not yet implemented but is planned for a future release. The current version has limited asynchronous capabilities. OCR support and a frontend UI are also listed as upcoming features.