eye_of_web  by MehmetYukselSekeroglu

Advanced OSINT platform for facial intelligence and security analysis

Created 6 months ago
299 stars

Top 88.8% on SourcePulse

GitHubView on GitHub
Project Summary

EyeOfWeb is an advanced, open-source platform for facial intelligence and security analysis, designed to combine Open Source Intelligence (OSINT) methodologies with deep learning-based biometric analysis. It serves as a powerful alternative to commercial face search engines for researchers, engineers, and security professionals, enabling sophisticated analysis of visual data from the web. The platform automates the process of crawling, detecting faces, generating unique vector embeddings, and indexing them for rapid searching and profiling.

How It Works

EyeOfWeb employs a hybrid database architecture, leveraging PostgreSQL for structured metadata (source, date, risk level) and Milvus for high-dimensional vector embeddings of detected faces. Visual data is autonomously crawled from various internet sources. Faces are detected and processed using the InsightFace AntelopeV2 model, which generates 512-dimensional embeddings. These embeddings are indexed in Milvus for efficient Approximate Nearest Neighbor (ANN) searches, enabling millisecond-level analysis of billions of faces, including 1:N identity searches, 1:1 comparisons, and social network analysis.

Quick Start & Requirements

  • Primary Install: Docker Compose is recommended for the fastest setup.
  • Prerequisites: Python 3.8+, Docker, Docker Compose, PostgreSQL (v13+), Milvus (v2.3+).
  • Recommended Hardware: Ubuntu 20.04+/Debian 11+, 8+ core CPU (AVX2), 16GB+ RAM, 250GB+ SSD.
  • GPU Support: Optional but recommended for performance; requires NVIDIA GPU with CUDA 11.x+ and 4GB+ VRAM. The Docker image uses CPU by default; GPU requires src/Dockerfile modification.
  • Links: Docker installation guide (doc/DOCKER.md), detailed documentation available in multiple languages.

Highlighted Details

  • Comprehensive Person Analysis: Advanced sociometric analysis to map social circles and reveal network patterns.
  • Deep Insight: Rapid analysis of face co-occurrence across images.
  • Multiple Search Modes: Supports image-based search, text/filter search, similar face search, and face similarity pair analysis.
  • AI Engine: Utilizes InsightFace AntelopeV2 for robust face detection, landmark identification, gender/age estimation, and vector embedding.
  • Security: Implements enterprise-grade security features including JWT authentication, bcrypt password hashing, CSRF protection, and rate limiting.
  • PDF Reporting: Generates professional, timestamped PDF reports for various analyses.
  • Playwright Crawler: Offers a 10x performance boost for web crawling with asynchronous multi-tab support.

Maintenance & Community

The project is marked as "Active Development" and is led by Project Owner/Lead Developer Mehmet Yüksel ŞEKEROĞLU, with contributions from Uğur POLAT (Academic Guidance) and Enes Ülker (Security Research). No specific community channels (like Discord or Slack) are listed.

Licensing & Compatibility

The project is licensed under the permissive MIT License. This allows for free use, modification, and distribution for both commercial and non-commercial purposes, provided the license and copyright notice are retained. No warranty is provided.

Limitations & Caveats

The default Docker image uses a CPU version of Torch for size optimization, requiring manual configuration for GPU acceleration. The project includes a strong legal disclaimer, emphasizing its development strictly for academic research, education, and legal security simulations, and disclaiming liability for any illegal or unethical use, particularly concerning privacy laws like GDPR and KVKK. Setup involves managing multiple complex services (Docker, PostgreSQL, Milvus).

Health Check
Last Commit

5 months ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
60 stars in the last 30 days

Explore Similar Projects

Starred by Chris Van Pelt Chris Van Pelt(Cofounder of Weights & Biases), Eric Zhu Eric Zhu(Coauthor of AutoGen; Research Scientist at Microsoft Research), and
1 more.

stealing-ur-feelings by noahlevenson

0%
924
AR experience revealing emotional surveillance dangers
Created 8 years ago
Updated 3 years ago
Feedback? Help us improve.