last30days-skill-cn  by Jesseovo

AI Agent for deep Chinese web research

Created 1 month ago
387 stars

Top 73.7% on SourcePulse

GitHubView on GitHub
Project Summary

This project provides an AI Agent skill for deep research into Chinese internet platforms, automatically gathering and analyzing content from the last 30 days across eight major platforms. It aims to deliver up-to-date, well-cited research reports rapidly, significantly reducing the need for API keys and manual data collection for users and AI agents focused on the Chinese digital landscape.

How It Works

The core innovation is the integration of the MediaCrawler engine, leveraging Playwright for browser automation. This approach drastically minimizes API key dependencies, allowing 7 out of 8 supported platforms to function without specific keys. It employs a three-tier data acquisition strategy: prioritizing configured APIs, falling back to the Playwright-based crawler, and finally using public interfaces as a last resort, ensuring maximum data availability.

Quick Start & Requirements

  • Primary Install: Clone the repository. Install core dependencies with pip install jieba playwright. Install Playwright browser binaries with playwright install chromium.
  • Prerequisites: Python 3.x, Playwright (for crawler mode). Optional API keys can be configured in ~/.config/last30days-cn/.env for enhanced stability or WeChat access.
  • Links: The project repository serves as the primary documentation source.

Highlighted Details

  • Supports 8 major Chinese platforms: Weibo, Xiaohongshu, Bilibili, Zhihu, Douyin, WeChat, Baidu, and Toutiao.
  • MediaCrawler integration enables 7/8 platforms to operate without API keys via Playwright automation.
  • Compatible with various AI Agent platforms including Cursor, Claude Code, OpenClaw, and Gemini CLI.
  • Features a smart scoring system weighting Relevance (45%), Recency (25%), and Engagement (30%).
  • Implements a three-tier fallback strategy: API → Crawler → Public API for robust data retrieval.

Maintenance & Community

The project is maintained by Jesse (@Jesseovo). No specific community channels (like Discord/Slack) or sponsorship details are mentioned in the README.

Licensing & Compatibility

This project is released under the MIT License, which is permissive for commercial use and integration into closed-source projects. However, the project's usage terms strictly prohibit commercial use of the scraped data or the tool itself for commercial data services, emphasizing its use for learning and research.

Limitations & Caveats

Users are explicitly warned that the project is for educational and research purposes only, and commercial use is forbidden. Strict adherence to Chinese laws, platform Terms of Service, and robots.txt is mandatory. Platform interfaces are subject to change, potentially impacting functionality, and users are advised to manage request frequency to avoid account bans. WeChat functionality primarily relies on API keys.

Health Check
Last Commit

1 month ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
1
Star History
111 stars in the last 30 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), Elvis Saravia Elvis Saravia(Founder of DAIR.AI), and
2 more.

OpenDeepSearch by sentient-agi

0.1%
4k
OpenDeepSearch: search tool for AI agents
Created 1 year ago
Updated 1 year ago
Feedback? Help us improve.