ThinkRAG is a locally deployable Retrieval-Augmented Generation (RAG) system designed for efficient Q&A over private knowledge bases on a laptop. It targets professionals, researchers, and students seeking an offline, privacy-preserving AI assistant, offering optimized handling of Chinese language data and flexible model integration.

How It Works

Built on LlamaIndex and Streamlit, ThinkRAG employs a modular architecture. It supports various LLMs via OpenAI-compatible APIs and local deployments through Ollama. For data processing, it utilizes SpacyTextSplitter for enhanced Chinese text segmentation and BAAI embedding/reranking models for improved relevance. The system offers a development mode with local file storage and an optional production mode leveraging Redis and LanceDB for persistent storage and vector indexing.

Quick Start & Requirements

• Install dependencies: pip3 install -r requirements.txt
• For offline use, download Ollama and LLMs (e.g., DeepSeek, Qwen, Gemma) via Ollama commands.
• Download embedding (e.g., BAAI/bge-large-zh-v1.5) and reranking models to the localmodels directory.
• Configure API keys via environment variables (e.g., OPENAI_API_KEY, DEEPSEEK_API_KEY) or the application interface.
• Run the system: streamlit run app.py
• Requires Python 3.x.
• Known Issue: Windows users may encounter issues; Linux or macOS is recommended.
• Dependency Note: Requires Ollama version 0.3.3 due to compatibility issues with newer versions.
• See docs/HowToDownloadModels.md for detailed model download instructions.

Highlighted Details

• Optimized for Chinese language processing with Spacy text splitter, Chinese prompt templates, and bilingual embedding models.
• Supports a wide range of LLMs, including popular Chinese providers like DeepSeek, Moonshot, and ZhiPu, alongside OpenAI and Ollama-compatible models.
• Offers both development (local file storage) and production (Redis, LanceDB) modes for flexible deployment.
• Enables local file uploads (PDF, DOCX, PPTX) and URL ingestion for knowledge base creation.

Maintenance & Community

The project is open-source and welcomes contributions. Links to community channels or roadmaps are not explicitly provided in the README.

Licensing & Compatibility

• License: MIT License.
• Compatible with commercial use and closed-source linking.

Limitations & Caveats

The system is not recommended for Windows users due to unresolved issues. A specific, older version of Ollama (0.3.3) is required for compatibility.