Local PDF and Markdown knowledge base with semantic search and AI-powered enrichment. It addresses the need for a unified, searchable, and locally-controlled repository for research papers, books, and notes, offering AI-driven metadata extraction and organization without API costs.

How It Works

This project employs a local-first architecture. It extracts text from PDF and Markdown files, then uses Ollama to generate embeddings for semantic search and perform AI-driven enrichment (e.g., title, summary, tags, concepts). Data is stored in libSQL, featuring an HNSW index for approximate nearest neighbor vector search and FTS5 for full-text search. This hybrid approach allows for both semantic understanding and fast keyword matching, with AI-extracted concepts organized via a SKOS taxonomy.

Quick Start & Requirements

• Primary install: Standalone binary (curl -fsSL https://raw.githubusercontent.com/joelhooks/pdf-brain/main/scripts/install.sh | bash) or via npm (npm install -g pdf-brain).
• Prerequisites: Ollama is required for embeddings and LLM operations. Installation varies by OS (macOS: brew install ollama, Linux: curl -fsSL https://ollama.com/install.sh | sh, Windows: download from ollama.com).
• Models: Download required embedding model (ollama pull mxbai-embed-large) and recommended enrichment model (ollama pull llama3.2:3b). Ensure Ollama server is running (ollama serve).
• Setup: Initialize the library (pdf-brain init) and add documents (pdf-brain add <path/to/document> [--enrich]).
• Resource Footprint: The libSQL database can grow significantly, with the HNSW vector index potentially reaching ~48GB for 500k chunks. Enrichment is CPU-intensive.
• Links: Official quick-start commands are provided in the README.

Highlighted Details

• Supports indexing and searching both PDF and Markdown files seamlessly.
• Fully local-first operation, ensuring data privacy and eliminating API costs.
• AI enrichment automatically extracts titles, summaries, tags, and suggests concepts.
• Organizes documents using a hierarchical SKOS taxonomy.
• Offers hybrid search combining vector similarity and full-text search capabilities.
• Integrates as an MCP server for AI coding assistants like Cursor.

Licensing & Compatibility

• License: MIT.
• Compatibility: The permissive MIT license generally allows for commercial use and integration with closed-source projects.

Limitations & Caveats

• Requires Ollama to be installed, configured, and running for core functionality.
• The vector index (HNSW) can lead to substantial disk space usage.
• AI enrichment is CPU-bound and can be slow for large document sets; GPU acceleration for Ollama is recommended.
• A specific libSQL quirk necessitates using SELECT COUNT(chunk_id) FROM embeddings for accurate embedding counts.