Zero-shot anomaly detection (ZSAD) for novel categories is addressed by AdaCLIP, which adapts the CLIP vision-language model. It introduces hybrid learnable prompts—static and dynamic—optimized with auxiliary data to significantly enhance ZSAD performance and generalization across diverse industrial and medical domains. This approach offers a robust solution for identifying anomalies in unseen categories, as presented in their ECCV 2024 paper.

How It Works

AdaCLIP integrates static and dynamic learnable prompts into the CLIP architecture. Static prompts provide initial adaptation for ZSAD tasks, while dynamic prompts offer per-image adaptation. This hybrid strategy, trained on auxiliary annotated anomaly detection datasets, allows CLIP to dynamically adjust its feature representations, leading to superior anomaly detection capabilities compared to standard ZSAD methods.

Quick Start & Requirements

Installation involves cloning the repository (https://github.com/caoyunkang/AdaCLIP.git) and running an install.sh script or manually setting up a Conda environment with Python 3.9.5 and specific PyTorch (1.10.1+cu111) and library versions (e.g., numpy==1.26.4, opencv-python). Users must download processed datasets and configure the DATA_ROOT path in config.py. Pre-trained weights are available for various dataset combinations. An optional Gradio app is provided for interactive use.

Highlighted Details

• Achieves state-of-the-art performance on 14 real-world anomaly detection datasets, outperforming existing ZSAD methods.
• Demonstrates strong generalization capabilities across different object categories and application domains (industrial and medical).
• Emphasizes the critical role of diverse auxiliary training data and optimized prompts for robust generalization.
• An online demo application is available for quick evaluation.

Maintenance & Community

The repository does not detail specific maintenance practices, contributor information, or community channels (e.g., Discord, Slack). It acknowledges inspiration from related projects like VAND-APRIL-GAN, AnomalyCLIP, and SAA.

Licensing & Compatibility

The provided README does not specify the software license or any compatibility notes for commercial use or integration with closed-source projects.

Limitations & Caveats

Training with half-precision (FP16) can lead to instability, necessitating multiple training runs to select the best performing model. Performance may exhibit minor variations compared to pre-trained weights due to potential version differences. The online demo's inference speed is notably slow (around 50 seconds per image) due to the absence of a GPU environment.