This project provides a robust, API-key-free Microservice Component Provider (MCP) for Google Search, integrating search, URL fetching, and academic paper extraction into a single, reliable tool. It addresses the common unreliability of existing search tools by offering a self-contained solution that handles CAPTCHAs and extracts valuable content, including academic PDFs, directly. The target audience includes engineers and researchers who need a dependable way to programmatically access Google Search results and academic literature without the overhead of API management or complex proxy setups.

How It Works

The MCP leverages Playwright with playwright-extra and stealth plugins for sophisticated browser automation, mimicking human behavior to avoid detection. It employs a multi-strategy SERP parser combined with geometric verification to intelligently filter out sponsored ads and knowledge panels, focusing on organic results. For content extraction, it integrates @llamaindex/liteparse for efficient PDF text extraction (supporting spatial parsing and multi-column layouts) and Mozilla Readability for HTML content, offering both full-text and abstract modes for cost-effective triage.

Quick Start & Requirements

• Primary Install: Use npx google-surf-mcp to run directly, or clone the repository (git clone ... && cd google-surf-mcp && npm install) for local development.
• Prerequisites: Node.js 18+ and Google Chrome (or Chromium) installed on the system.
• Setup: The first tool call automatically bootstraps a warm Chrome profile. Manual bootstrapping (npm run bootstrap) is available if needed. Environment variables like CHROME_PATH can override default paths.

Highlighted Details

• Unified Functionality: Replaces separate search, fetch, and academic-search MCPs with a single tool.
• Academic PDF Extraction: Supports direct extraction from arXiv, bioRxiv, Nature, OpenReview, NeurIPS, JMLR, PMLR, Springer, and PubMed (via PMC).
• Content Modes: Offers abstract mode (~1500 chars) for quick relevance checks and full mode for complete article bodies.
• SERP Filtering: Intelligently drops sponsored ads and knowledge panels using geometric verification.
• CAPTCHA Recovery: Features four robust modes: OS notification (default), visible Chrome (SURF_HEADLESS=false), remote debugging (SURF_REMOTE_DEBUG), and fail-fast cloud mode (SURF_CLOUD_MODE).
• Performance: Achieves ~1.5s per query for sequential search and ~3s wall time for five search_extract operations in abstract mode on a 1Gb/s connection.

Maintenance & Community

The project includes built-in self-healing mechanisms, such as runtime strategy reordering and optional LLM-assisted repair, indicating an active focus on maintaining robustness against website changes. No specific community channels (like Discord/Slack) or notable contributors are listed in the provided text.

Licensing & Compatibility

• License: MIT.
• Compatibility: The MIT license permits broad use, including commercial applications and linking within closed-source projects, without significant restrictions.

Limitations & Caveats

While CAPTCHAs are handled, human intervention may be required in default local setups, potentially impacting fully unattended automation unless SURF_CLOUD_MODE (fail-fast) is used. Serverless or headless environments require specific configuration (e.g., SURF_REMOTE_DEBUG, SURF_CLOUD_MODE). The tool relies on a local Chrome installation, and occasional stale selectors might necessitate profile refreshes or manual intervention.