This project provides an LLM-powered assistant for public opinion analytics, integrating real-time data from 15 platforms and 26 lists. It targets engineers, researchers, and power users by offering a conversational interface for hot search queries, topic clustering, and sentiment analysis, with multi-channel push notifications for actionable insights.

How It Works

The system comprises a decoupled crawler cluster and an analysis agent. It ingests real-time data from numerous platforms, processing it through a large language model (LLM), with Huawei's Pangu LLM recommended for its strong Chinese text comprehension and local deployment capabilities. Users interact via a frontend to perform conversational searches, topic clustering, and sentiment analysis. Results can be pushed through various channels, including email, WeChat, and Telegram, based on analyzed news content, even from video sources.

Quick Start & Requirements

Installation involves setting up a Python virtual environment and running pip install -r requirements.txt. Key prerequisites include a MySQL database, a compatible web browser (Chrome/Edge), and correctly configured browser drivers (e.g., chromedriver, msedgedriver) accessible via the system's PATH. Users must also configure LLM API parameters or set up a local LLM like Pangu. Detailed instructions are provided for browser driver setup, database initialization, and parameter configuration for both the crawler and analysis systems.

Highlighted Details

• Leverages LLMs for advanced natural language understanding, enabling conversational queries and nuanced sentiment analysis, including handling sarcasm and implicit expressions in Chinese text.
• Capable of extracting and analyzing information from news details, including video content.
• Supports multi-channel push notifications for critical updates via email, WeChat (group/personal), and Telegram bots.
• Offers local deployment options for recommended LLMs like Pangu, enhancing data privacy and control.

Maintenance & Community

The project is maintained by [huangmingqi]. Specific details regarding community channels (e.g., Discord, Slack), active contributors, sponsorships, or a public roadmap are not detailed in the provided documentation. The author requests stars for continued development.

Licensing & Compatibility

The project is licensed under the MIT license. Its dependencies primarily use permissive licenses (MIT, BSD, Apache 2.0), making it compatible with commercial use and integration into closed-source projects.

Limitations & Caveats

This project is explicitly designated for learning, research, and exchange purposes only, and is not intended as professional advice or service. Users are solely responsible for ensuring their activities comply with all relevant laws, website terms of service, and ethical data handling practices. Setup requires significant technical effort, particularly for browser driver and database configuration. The effectiveness of analysis is dependent on the chosen LLM and its configuration.