This project provides an API gateway that enables Large Language Models (LLMs) to access the internet for search, news, and web content summarization. It targets users of LLM APIs who want to integrate web search capabilities without modifying their existing clients or LLM keys. The primary benefit is adding real-time information retrieval to LLMs.

How It Works

The project acts as a proxy, intercepting LLM requests and determining if an internet search is necessary based on the user's input. If a search is triggered, it utilizes various search engine APIs (Google, Bing, DuckDuckGo, SearXNG, etc.) to fetch results. It can then optionally crawl and summarize the content of these results. The system supports multiple LLM providers including OpenAI, Azure OpenAI, Groq, Gemini, and Moonshot, with varying levels of support for streaming output.

Quick Start & Requirements

• Local Deployment: cd api && nohup node index.js > output.log 2>&1 & (Listens on port 3014)
• Prerequisites: Node.js. API keys for chosen search services and LLM providers are required.
• Deployment Options: Zeabur (one-click), Cloudflare Worker, Vercel, local Node.js.
• Documentation: English, 简体中文
• Community: Discord

Highlighted Details

• Supports multiple LLM providers (OpenAI, Azure OpenAI, Groq, Gemini, Moonshot).
• Integrates with various search engines: Google, Bing, DuckDuckGo, SearXNG, Search1API, SerpApi, Serper.
• Offers flexible deployment options including Cloudflare Workers and Zeabur one-click deployment.
• Allows custom authorization keys for user requests.

Maintenance & Community

The project is actively updated, with recent versions (V0.2.6 as of April 25, 2024) adding support for SearXNG and Moonshot streaming. A Discord channel is available for user interaction.

Licensing & Compatibility

The repository does not explicitly state a license in the README. This may pose compatibility issues for commercial or closed-source use.

Limitations & Caveats

The Vercel deployment option has limitations, including no support for streaming output and a 10-second response time limit, which may result in a poor user experience. The project is under active development, and specific features like streaming for certain providers or search services might be experimental or incomplete.