firecrawl-observer  by firecrawl

Website change monitoring with AI-powered analysis

Created 3 months ago
327 stars

Top 83.3% on SourcePulse

GitHubView on GitHub
Project Summary

This project provides a robust website monitoring application for users to track changes on web pages and receive intelligent notifications. Built with Next.js, Convex, and the Firecrawl API, it targets individuals and businesses needing to stay updated on website content, offering AI-powered analysis to filter out noise and flexible notification options.

How It Works

The application leverages Convex for its backend, providing real-time data synchronization and a scalable database. Firecrawl API is used for website crawling and change detection, enabling monitoring of single pages or entire websites. AI models (OpenAI, Anthropic, Google, Moonshot) can be integrated for intelligent analysis, identifying meaningful changes and reducing notification noise. API keys are securely stored using AES-256-GCM encryption.

Quick Start & Requirements

  • Install: npm install
  • Prerequisites: Node.js 18+, npm, Convex account, Firecrawl API key. Optional: Resend API key, AI API key.
  • Setup: Requires running npx convex dev, npx @convex-dev/auth, and npm run dev.
  • Docs: Firecrawl Documentation, Convex Documentation

Highlighted Details

  • AI-powered analysis for intelligent change detection.
  • Flexible notification options: email, webhooks, or dashboard-only.
  • Secure, encrypted storage for API keys (AES-256-GCM).
  • Real-time updates and a responsive UI with dark mode.

Maintenance & Community

The project is maintained by mendableai. Links to community support are not explicitly provided in the README.

Licensing & Compatibility

The project is licensed under the MIT License, permitting commercial use and closed-source linking.

Limitations & Caveats

The minimum check interval for monitoring is 60 minutes. While optional AI API keys enhance functionality, they are not required for basic website monitoring.

Health Check
Last Commit

2 months ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
22 stars in the last 30 days

Explore Similar Projects

Feedback? Help us improve.