This project provides a specialized web search MCP server designed to enhance AI coding tools and agents by delivering richer, more contextually complete web content. It addresses the limitation of standard search tools that often return only snippets or links, requiring AI to perform additional, inefficient scraping. By retrieving full conversations and structured data from sources like StackExchange and GitHub Issues, and parsing any webpage in real-time, it aims to significantly improve the quality and efficiency of AI-generated code.

How It Works

Kindly Web Search leverages direct API integrations with platforms like StackExchange, GitHub Issues, arXiv, and Wikipedia to fetch structured content, including questions, answers, comments, and metadata, optimized for Large Language Models. For broader web content, it employs a headless Chromium browser via nodriver for real-time page parsing. This approach ensures that AI assistants receive comprehensive information in a single call, eliminating the need for secondary scraping and reducing token waste. It supports multiple search providers (Serper, Tavily, SearXNG) with intelligent fallback mechanisms.

Quick Start & Requirements

• Primary install/run command: Use uvx to install and run the MCP server:

  uvx --from git+https://github.com/Shelpuk-AI-Technology-Consulting/kindly-web-search-mcp-server kindly-web-search-mcp-server start-mcp-server
  

• Prerequisites:
  • A search provider API key: SERPER_API_KEY (recommended), TAVILY_API_KEY, or SEARXNG_BASE_URL (for self-hosted SearXNG).
  • A Chromium-based browser (Chrome, Chromium, Edge, Brave) installed locally for universal HTML page content extraction.
  • GITHUB_TOKEN (optional, but recommended for enhanced GitHub Issue/Discussion parsing).
  • Python 3.13+ (Python 3.14 is supported with potential caveats regarding onnxruntime wheels).
• Estimated setup time: The initial uvx environment build can take 30-60 seconds. Browser installation is a standard OS task.
• Relevant pages: The repository is hosted on GitHub.

Highlighted Details

• Aims to improve AI-generated code quality by 15–20%.
• Retrieves full conversations (questions, answers, comments, metadata) from specialized sources.
• Performs real-time parsing of arbitrary webpages using a headless browser.
• Supports Serper, Tavily, and SearXNG search providers with intelligent fallback.
• Consolidates functionality, reducing the need for separate web scraping or specialized MCP servers.

Maintenance & Community

The project encourages community engagement through GitHub stars as a primary motivator. No specific details on active maintainers, sponsorships, or dedicated community channels (like Discord or Slack) are provided in the README.

Licensing & Compatibility

The specific open-source license is not explicitly stated in the provided README. Compatibility for commercial use or linking with closed-source projects is not detailed.

Limitations & Caveats

Universal HTML page content extraction is dependent on a locally installed Chromium-based browser; specialized sources function without it. Python 3.14 support has noted limitations. Remote deployments via Docker require careful security configuration as the default HTTP endpoint is unauthenticated and unencrypted. Timeout issues during content retrieval may necessitate adjusting environment variables like `KINDLY_TOOL_TOTAL_TIMEOUT_SECONDS`.