SentrySearch enables semantic search over video footage, allowing users to find specific moments via natural language queries. It targets users with large video archives (e.g., dashcam, surveillance), offering precise, sub-second clip retrieval without manual scrubbing or transcription.

How It Works

The system splits MP4 videos into overlapping chunks, embedding each using Google's Gemini Embedding API or a local Qwen3-VL model. These embeddings are stored in ChromaDB. Text queries are embedded and matched against video embeddings, with the top result automatically trimmed. This approach uniquely embeds raw video pixels directly, making text queries comparable to video content at the vector level for efficient semantic search.

Quick Start & Requirements

Installation uses uv. After cloning and uv sync, initialize with sentrysearch init (sets API key). Index footage via sentrysearch index /path/to/footage, and search with sentrysearch search "query". Prerequisites include Python 3.11+ and ffmpeg. The Gemini backend requires an API key; the local backend benefits from a GPU (CUDA/Metal).

Highlighted Details

• Direct Video Embedding: Utilizes Gemini Embedding 2 or Qwen3-VL to embed video content directly, bypassing transcription.
• Local Backend: Supports offline operation with Qwen3-VL (including quantized versions) for privacy and no API costs, though GPU is recommended for speed.
• Tesla Metadata Overlay: Integrates with Tesla dashcam footage (firmware 2025.44.25+, HW3+) to overlay speed, location, and time.
• Cost/Performance Optimizations: Default settings include preprocessing (downscaling, frame rate reduction) and still-frame skipping to minimize Gemini API usage and speed up indexing.

Maintenance & Community

No specific details regarding maintainers, community channels, or project roadmap were found in the provided README.

Licensing & Compatibility

The repository's license is not explicitly stated. The tool is compatible with MP4 video files. The Tesla overlay feature has specific hardware/firmware requirements.

Limitations & Caveats

Still-frame detection is heuristic and may miss subtle motion or index static segments. Search accuracy can be affected by chunk boundaries. The Gemini Embedding 2 API is in preview, subject to change. Indices from Gemini and local backends are incompatible, requiring re-indexing when switching.