firecrawl-mcp-server  by mendableai

MCP server adds web scraping to LLM clients

created 7 months ago
3,988 stars

Top 12.5% on sourcepulse

GitHubView on GitHub
Project Summary

This project provides a Model Context Protocol (MCP) server that integrates Firecrawl's web scraping capabilities into LLM clients like Cursor and Claude. It enables LLMs to perform web scraping, crawling, searching, and data extraction, enhancing their ability to research and interact with web content.

How It Works

The server acts as a bridge, translating LLM requests into Firecrawl API calls. It supports various scraping and research tasks, including single URL scraping, batch scraping, web crawling, and deep research using LLM analysis. It leverages Firecrawl's JS rendering, automatic retries, rate limiting, and content filtering, offering both cloud and self-hosted Firecrawl instance support.

Quick Start & Requirements

  • Install/Run: npx -y firecrawl-mcp
  • Prerequisites: Node.js, npm. A Firecrawl API key is required for cloud API usage.
  • Setup: Minimal setup, primarily requiring an API key and configuration within the target LLM client (e.g., Cursor, VS Code).
  • Docs: Cursor MCP Server Configuration Guide

Highlighted Details

  • Supports scraping with JS rendering, URL discovery, and crawling.
  • Offers batch scraping with rate limiting and automatic retries.
  • Includes tools for web search, deep research, and structured data extraction via LLMs.
  • Provides comprehensive logging and robust error handling with backoff strategies.

Maintenance & Community

The project acknowledges contributions from @vrknetha and @cawstudios for initial implementation. Integrations and hosting support are noted from MCP.so and Klavis AI.

Licensing & Compatibility

  • License: MIT License.
  • Compatibility: Permissive license suitable for commercial use and integration into closed-source applications.

Limitations & Caveats

The README mentions specific version requirements for Cursor integration (0.45.6+). Some configurations might require adjustments for Windows environments.

Health Check
Last commit

1 week ago

Responsiveness

1 week

Pull Requests (30d)
3
Issues (30d)
2
Star History
1,143 stars in the last 90 days

Explore Similar Projects

Starred by Tobi Lutke Tobi Lutke(Cofounder of Shopify), Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and
7 more.

firecrawl by mendableai

1.9%
44k
API service for turning websites into LLM-ready data
created 1 year ago
updated 1 day ago
Feedback? Help us improve.