This project provides an AI-maintained knowledge base system built on Obsidian, inspired by Andrej Karpathy's LLM Wiki pattern. It addresses the challenge of creating a compounding, structured personal knowledge base by automating the ingestion, organization, and cross-referencing of raw information sources. Aimed at engineers, researchers, and power users, it offers a "second brain" solution that enhances knowledge discovery and retention through continuous AI-driven curation and user interaction, outperforming traditional RAG systems for Q&A at scale.

How It Works

The system eschews RAG's on-demand processing for proactive knowledge compilation, ingesting raw inputs (URLs, text, PDFs) into structured wiki pages with automatic [[wikilink]] generation. Key features include bias checks (counter-arguments, data gaps) and master index maintenance. Queries are answered by synthesizing information from compiled pages, with each answer filed back into the wiki, creating a compound loop that continuously enriches the knowledge base for structured, readily accessible information.

Quick Start & Requirements

• Primary install: git clone https://github.com/shannhk/llm-wikid.git my-wiki, cd my-wiki, open as Obsidian vault.
• Run agent: claude --dangerously-skip-permissions (or compatible agent like OpenClaw, Hermes, Codex).
• Prerequisites: Obsidian, a compatible LLM agent capable of reading markdown and running shell commands. Optional tools for ingest include yt-dlp, scrapling, summarize (PDF extraction), and X API keys. Optional search tool qmd (npm install -g @tobilu/qmd) and last30days skill are recommended.
• Links: Repo: https://github.com/shannhk/llm-wikid.git.

Highlighted Details

• Quality Controls: Implements bias checks (counter-arguments, data gaps), validation gates (explored: false until manually verified), confidence tagging (high, medium, low, uncertain), source tracing for every claim, and stub creation for undefined [[wikilinks]].
• Git Sync: All changes are committed and pushed to Git, enabling version control, reversibility (git revert), multi-agent collaboration, and remote access.
• Claude Dispatch: Automated remote ingest via scheduled triggers (e.g., cron) processes new sources overnight.
• Compound Loop: User interactions and filed answers continuously enrich the knowledge base, making it smarter over time.
• Search Integration: Supports qmd for hybrid BM25/vector search with LLM re-ranking, usable as a native tool via MCP.

Maintenance & Community

• Built by @shannholmberg. Credits Andrej Karpathy (LLM Wiki pattern), hooeem (LLM Knowledge Base course), Tobi Lutke (qmd), and mvanhorn (last30days). No explicit community links (Discord/Slack) are provided.

Licensing & Compatibility

• License: The repository's license is not specified in the README, posing a significant adoption risk, particularly for commercial use.
• Compatibility: Designed for LLM agents supporting markdown and shell commands. Requires Obsidian (v1.9.10+ for dashboard features).

Limitations & Caveats

• The absence of a clear license is a major blocker for adoption. Functionality relies heavily on a compatible LLM agent and may require installing and configuring additional external tools and API keys. While scaling strategies are outlined, the file-based approach may face performance limits at extreme scales.