YaCy Expert aims to create a question-answering search engine by combining Large Language Models (LLMs) with Retrieval Augmented Generation (RAG). It targets users with large text corpora, such as those acquired through YaCy web crawling, enabling them to build domain-specific "expert" chatbots. The project leverages LLMs for semantic understanding and RAG for context delivery, moving beyond traditional keyword-based search.

How It Works

The system architecture consists of a web interface acting as a wrapper for two backend services: an LLM (inference engine) and a RAG system (knowledge base). The LLM is designed as a drop-in replacement for the OpenAI chat API, utilizing llama.cpp. The RAG system is embedded within the YaCy Expert web interface's backend and uses Faiss for efficient vector similarity search on user-provided data dumps. This approach allows for semantically relevant context retrieval from large text corpora to augment LLM responses.

Quick Start & Requirements

• Installation: Requires Python, a server for the LLM (native or Docker), and one or more YaCy index dump files.
• Hardware: Recommends significant RAM (64GB+) for LLM and vector index. A proof-of-concept for Raspberry Pi is in development.
• Data Indexing: Involves exporting YaCy data (.jsonlist), placing it in the knowledge directory, and running python3 knowledge_indexing.py . (approx. 1 hour per 10,000 entries). Custom BERT models can be specified via .ini files.
• LLM Backend: Can be configured via OpenAI API or by self-hosting an LLM using llama.cpp (Docker recommended).
• Running: Start the YaCy Expert backend server and access the chat web page.
• Docs: YaCy Expert README

Highlighted Details

• Leverages llama.cpp for OpenAI API compatibility, enabling self-hosted LLMs.
• Utilizes Faiss for efficient vector similarity search, forming the RAG knowledge base.
• Designed to process YaCy data dumps (WARC, ZIM) for domain-specific knowledge.
• Inspired by "Expert Systems" AI concepts, separating inference and knowledge base.

Maintenance & Community

• Project is hosted on GitHub under the yacy organization.
• No specific community links (Discord/Slack) or roadmap details are provided in the README.

Licensing & Compatibility

• The README does not explicitly state a license. The parent yacy project is typically licensed under GPLv2.

Limitations & Caveats

• Indexing large datasets (millions of entries) can take days or weeks.
• The project is described as a "stub" for installation, implying it may require significant setup effort.
• Performance is highly dependent on hardware, particularly RAM and GPU availability for the LLM and vector index.