LLM_Web_search  by mamei16

WebUI extension for LLM web search using DuckDuckGo or SearXNG

created 1 year ago
253 stars

Top 99.5% on sourcepulse

GitHubView on GitHub
Project Summary

This extension for oobabooga/text-generation-webui enables local Large Language Models (LLMs) to perform web searches. It targets users who want to augment their LLM's knowledge base with real-time information from the internet, providing a mechanism for LLMs to access and process external data.

How It Works

The extension intercepts LLM output for specific commands (e.g., Search_web("query")) using regular expressions. It then utilizes DuckDuckGo (or a SearXNG instance) to perform the search, retrieving results. An ensemble of a dense embedding model and Okapi BM25 or SPLADE is employed to extract relevant snippets from the search results, which are then appended to the LLM's output. This approach allows for contextually relevant information retrieval, enhancing the LLM's ability to answer questions with up-to-date data.

Quick Start & Requirements

  • Installation: Install via the "Session" tab in text-generation-webui using "Install or update an extension."
  • Dependencies: Requires installation via pip (potentially using unofficial faiss-cpu) or manual update of the conda environment using the provided environment.yml.
  • Launch: Enable the extension in the "Session" tab or run python server.py --extension LLM_Web_search.
  • Documentation: text-generation-webui Wiki Extensions

Highlighted Details

  • Supports both "Simple search" (highlighted snippets) and "Full search" (entire web pages).
  • Offers Okapi BM25 (CPU-based, fast) and SPLADE (GPU-accelerated, potentially more accurate) as keyword retrieval backends.
  • Provides multiple chunking methods: character-based, semantic, and token classification-based.
  • Recommends specific LLMs for users with ≤ 12 GB VRAM, including Llama-3.1-8B-instruct and Mistral-Nemo-Instruct-2407.

Maintenance & Community

  • The project is an extension for oobabooga/text-generation-webui, benefiting from its community and development.

Licensing & Compatibility

  • The README does not explicitly state a license for this extension. Compatibility with text-generation-webui implies it's intended for use within that ecosystem.

Limitations & Caveats

  • The "easy" installation method relies on faiss-cpu, which may not work on all systems.
  • Reading full web pages may exceed LLM context length.
  • SPLADE's performance is dependent on available VRAM and batch size tuning.
Health Check
Last commit

3 days ago

Responsiveness

1 day

Pull Requests (30d)
7
Issues (30d)
1
Star History
16 stars in the last 90 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), Simon Willison Simon Willison(Author of Django), and
1 more.

Lumos by andrewnguonly

0.1%
2k
Chrome extension for local LLM web RAG co-piloting
created 1 year ago
updated 6 months ago
Feedback? Help us improve.