This repository provides the official code for a TIP 2025 paper on Vision-Language Models (VLMs) for enhanced nighttime object detection. It addresses the challenge of detecting objects in low-light and adverse conditions by introducing a Consistency Sampler (CCCS) and a Hallucination Feature Generator (VLMOE). The project benefits researchers and engineers seeking to improve the robustness and accuracy of object detection systems in challenging visual environments.

How It Works

The core approach integrates VLMs with novel components: CCCS for feature refinement via clustering and ranking, and VLMOE for generating synthetic features to augment training data. VLMOE leverages CLIP and employs an Orthogonal Projection Loss for feature learning and KL divergence to ensure consistency between real and generated features. This dual strategy aims to improve detection performance by better handling the visual ambiguities inherent in nighttime imagery.

Quick Start & Requirements

• Installation: Requires Python 3.7, PyTorch 1.7.1+cu111, and CUDA 10.2. Setup involves conda environment creation, cloning the repo, and installing numerous dependencies including cocoapi, pynndescent, opencv-python, and CLIP.
• Prerequisites: Nvidia Tesla V100 single GPU recommended. Datasets: SHIFT, BDD100k, FLIR.
• Resources: Pre-trained weights and COCO-format annotations are available via Google Drive.

Highlighted Details

• Official implementation for the TIP 2025 paper "Vision-Language Models Empowered Nighttime Object Detection with Consistency Sampler and Hallucination Feature Generator."
• Includes adaptations for the Diverse Weather Dataset (Diverse101) under source-only cross-domain and domain generalization settings.
• Provides converted COCO-format annotations and pre-trained model weights.

Maintenance & Community

No specific details regarding community channels (e.g., Discord, Slack), active maintainers, sponsorships, or a public roadmap are present in the provided README.

Licensing & Compatibility

The README does not specify a software license. This lack of information prevents an assessment of compatibility for commercial use or integration into closed-source projects.

Limitations & Caveats

The project mandates specific, older versions of Python (3.7) and PyTorch (1.7.1), potentially leading to compatibility issues with modern development environments. The associated paper is noted as "Under Review," indicating the project may still be in an active research and development phase. The setup process is complex, requiring careful management of dependencies and datasets.