This project provides a system for searching video content using image queries, specifically designed for anime but applicable to any MP4 video. It leverages a modified LIRE (Lightweight Image Representation Extraction) library integrated with Apache Solr for efficient indexing and retrieval, enabling accurate scene lookups down to the millisecond.

How It Works

The system indexes video frames by extracting image features using LIRE. These features are then stored in Apache Solr cores. When a query image is provided, its features are extracted and used to search across the Solr index. The approach uses Locality-Sensitive Hashing (LSH) to group similar frames, allowing for fast searches by prioritizing less populated clusters. This method is optimized for speed and scalability, as demonstrated by its use in the trace.moe service.

Quick Start & Requirements

• Install: Clone the repository, run npm install.
• Prerequisites: Linux (Fedora tested), Node.js 12/14, ffmpeg, Java 1.8, podman-compose.
• Configuration: Requires setting up .env with database, Solr, and RabbitMQ connection details. Solr home directory needs 777 permissions.
• Setup: Involves starting Docker containers (podman-compose up -d), creating Solr cores (npm run create-core), and running workers for hashing (npm run hash) and loading (npm run load).
• Search: Execute node src/search.js /path/to/your/image.jpg.
• Links: trace.moe (demonstrates the technology).

Highlighted Details

• Achieves accurate scene lookups to 0.01s.
• Scalable to search 700 million frames in under 3 seconds.
• Supports distributed workers for indexing and frame loading.
• Includes features like frame deduplication and automatic indexing of new videos via a watch folder.

Maintenance & Community

The project is maintained by soruly. Further community or maintenance details are not explicitly provided in the README.

Licensing & Compatibility

The README does not specify a license. Compatibility for commercial use or closed-source linking is not mentioned.

Limitations & Caveats

The system requires specific Linux distributions and versions of Node.js and Java. RabbitMQ heartbeat intervals may need adjustment for long-running tasks. The setup involves multiple services (MariaDB, RabbitMQ, Solr) managed via podman-compose, requiring familiarity with container orchestration.