This project provides an intelligent video surveillance early warning system using multimodal visual models. It's designed for security professionals and researchers needing to automate anomaly detection in video feeds, offering real-time alerts and customizability for various scenarios.

How It Works

The system processes video input, extracting frames for a multimodal analyzer. This analyzer generates textual descriptions of the video content, which are then fed into an anomaly detection module. Detected anomalies trigger an early warning service that can push alerts via WebSocket, save abnormal video clips, and store historical data. An optional RAG system can vectorize video descriptions and historical summaries into a vector database for intelligent Q&A.

Quick Start & Requirements

• Install dependencies: pip install -r requirements.txt
• Configure config.py with model API addresses and keys.
• Run with local video or RTSP stream: python video_server.py --video_source "<path_or_rtsp_url>"
• WebSocket endpoints: ws://localhost:16532/alerts and ws://localhost:16532/video_feed
• Requires Python, and potentially a vector database (e.g., Milvus) for RAG functionality.

Highlighted Details

• Supports local video files and RTSP streams.
• Generates scene descriptions and detects suspicious behaviors.
• Real-time WebSocket push for alerts and video streams.
• Saves abnormal video segments and detailed descriptions.
• Optional RAG integration for intelligent Q&A on historical data.

Maintenance & Community

No specific contributors, sponsorships, or community links (Discord/Slack) are mentioned in the README.

Licensing & Compatibility

The README does not specify a license. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

The open-source version focuses on core monitoring and alarm functions; RAG and large model APIs require self-configuration. The README does not mention specific model requirements beyond API configuration.