Discover and explore top open-source AI tools and projects—updated daily.
fivetakuAdaptive web scraper for resilient data extraction
New!
Top 56.2% on SourcePulse
<2-3 sentences summarising what the project addresses and solves, the target audience, and the benefit.> insane-search is a Claude Code plugin designed to bypass common web blocking mechanisms like WAFs, CAPTCHAs, and login walls, enabling users to access content from stubborn websites. It targets users within the Claude Code ecosystem who frequently encounter access restrictions, offering a seamless, no-configuration solution that retrieves data where other tools fail. The primary benefit is reliable web content access without requiring API keys or complex setup.
How It Works
<2-4 sentences on core approach / design (key algorithms, models, data flow, or architectural choices) and why this approach is advantageous or novel.>
The core of insane-search is a 5-phase adaptive scheduler that intelligently escalates probing techniques. It begins with lightweight methods and progresses through TLS fingerprint impersonation with sophisticated identity spoofing (including cookie warming and referrer chains) to a full browser environment using Playwright. This multi-phase approach allows it to discover hidden APIs by monitoring network traffic and adapt to site-specific challenges. Its novelty lies in its "never give up" philosophy: it auto-installs missing dependencies like curl_cffi and yt-dlp transparently, and it doesn't pre-judge sites as inaccessible, ensuring maximum retrieval success.
Quick Start & Requirements
/plugin marketplace add https://github.com/fivetaku/gptaku_plugins.git), install insane-search (/plugin install insane-search), and restart Claude Code.curl_cffi, feedparser, and yt-dlp are auto-installed on demand. Optional enhancements include gh CLI and Playwright.Highlighted Details
curl_cffi, yt-dlp) when needed.Maintenance & Community
The provided README does not detail specific contributors, sponsorships, or community channels such as Discord or Slack.
Licensing & Compatibility
Limitations & Caveats
The tool's effectiveness is limited by sites requiring explicit authentication ("authentication required"). It functions as a method-selection layer rather than a traditional scraper, relying on the Claude Code environment for execution.
6 days ago
Inactive