This package provides an advanced Retrieval-Augmented Generation (RAG) system for querying local documents using Large Language Models (LLMs). It targets developers and researchers seeking a robust, configurable solution for document Q&A, offering enhanced features over basic RAG implementations. The system aims to improve document parsing, search relevance, and user interaction through a flexible YAML configuration.

How It Works

The system employs a hybrid search approach, combining dense embeddings (from models like Sentence-Transformers, Instructor, OpenAI) with sparse embeddings generated by SPLADE. This dual strategy aims to capture both semantic meaning and keyword relevance for more accurate retrieval. It supports incremental indexing, efficient parsing of various document formats (Markdown, PDF, DOCX), and integrates with LiteLLM and Ollama for broad LLM compatibility, including locally hosted models. Advanced features like HyDE, re-ranking, and multi-querying are included to further refine search quality.

Quick Start & Requirements

• Install via pip install pyllmsearch.
• Requires Python 3.8+.
• Supports local LLMs (via Ollama/LiteLLM) and OpenAI API.
• Documentation: Browse Documentation

Highlighted Details

• Advanced RAG features: Hybrid search (dense + SPLADE sparse), HyDE, re-ranking, multi-querying.
• Broad LLM and embedding model support via LiteLLM and HuggingFace.
• Incremental parsing and embedding for document bases up to several gigabytes.
• Parsers for Markdown, PDF (MuPDF), DOCX, with support for Unstructured for other formats.
• MCP server for integration with clients like Cursor, Windsurf, VSCode GH Copilot.

Maintenance & Community

• Developed by snexus.
• Links to demo and documentation are provided.

Licensing & Compatibility

• License: MIT.
• Compatible with commercial and closed-source applications.

Limitations & Caveats

The README notes that enabling HyDE can significantly alter result quality and advises users to consult the associated paper. Image parsing requires the Gemini API, which may incur costs.