ChatPDF is a locally deployed Retrieval Augmented Generation (RAG) system designed for question-answering over documents. It targets users who want to leverage local Large Language Models (LLMs) and embedding models for private, efficient document interaction, supporting various file types and offering advanced RAG techniques.

How It Works

This project implements a RAG pipeline using local LLMs, embedding models, and reranker models, with an emphasis on pure native implementation without external agent libraries. It supports GraphRAG for relational knowledge retrieval and offers flexibility in choosing LLMs (OpenAI, Deepseek, Ollama, etc.) and embedding models (text2vec, HuggingFace, Sentence-Transformers). Optimizations include Chinese chunk splitting, sentence embedding, BM25 with Jieba for keyword matching, and a reranker module to refine candidate chunks, enhancing accuracy.

Quick Start & Requirements

• Install dependencies: pip install -r requirements.txt
• Recommended: Use WSL on Windows. Ensure CUDA is installed if not using CPU-only inference.
• Run RAG: CUDA_VISIBLE_DEVICES=0 python rag.py
• Launch Gradio Web UI: python webui.py (accessible at http://localhost:8082)
• GraphRAG Demo: python graphrag_demo.py (requires OPENAI_API_KEY or Ollama setup)
• Supported file formats: PDF, DOCX, Markdown, TXT.

Highlighted Details

• Supports local LLMs and embedding models for privacy and cost savings.
• Implements GraphRAG for enhanced relational context retrieval.
• Features optimized chunking, embedding, retrieval (BM25 + semantic similarity), and reranking for improved accuracy.
• Allows context window expansion for retrieved chunks.
• Can utilize large context LLMs (e.g., 200k tokens) and custom RAG-tuned models.
• Provides a Gradio-based web UI with streaming dialogue.

Maintenance & Community

• Contact: xuming624@qq.com or WeChat (xuming624).
• Open to contributions and code improvements.
• Related project: shibing624/MedicalGPT for LLM training.

Licensing & Compatibility

• License: Apache License 2.0.
• Permissive for commercial use, requiring attribution to ChatPDF and inclusion of the license.

Limitations & Caveats

The project is described as "rough code" and welcomes improvements. While it supports various LLMs and embedding models, performance and compatibility may vary depending on the specific local models chosen.