GeoCLIP is a PyTorch implementation for worldwide image geo-localization, inspired by OpenAI's CLIP. It addresses the challenge of accurately determining the geographical location of an image by aligning visual features with precise GPS coordinates. The project is targeted at researchers and developers working on computer vision, geospatial AI, and location-aware systems, offering state-of-the-art performance and a versatile pre-trained location encoder.

How It Works

GeoCLIP employs a contrastive learning approach, similar to CLIP, to match image-GPS pairs. It models the Earth as a continuous function, enabling its location encoder to learn semantically rich, CLIP-aligned features. This method leverages a large dataset (MP-16, 4.7M images) to associate distinct visual cues with specific global locations, resulting in a powerful representation for geo-localization tasks. The location encoder's architecture is designed for generalization, making it suitable for use as a pre-trained component in other geo-aware neural networks.

Quick Start & Requirements

• Installation: pip install geoclip or clone the repository and run python setup.py install.
• Prerequisites: PyTorch.
• Usage: The README provides Python code snippets for both inference (predicting GPS from an image) and using the pre-trained location encoder to generate GPS embeddings.

Highlighted Details

• Achieves state-of-the-art results on benchmark datasets: Im2GPS3k, YFCC26k, GWS15k, and Geo-Tagged NUS-Wide Dataset.
• Demonstrates effectiveness as a pre-trained GPS encoder, improving multi-class classification accuracy on geo-aware tasks.
• The location encoder can be used independently for GPS-only multi-class classification, achieving top performance.

Maintenance & Community

The project is an official implementation of a NeurIPS 2023 paper. Further community or maintenance details are not explicitly provided in the README.

Licensing & Compatibility

The README does not specify a license. The project incorporates code from Joshua M. Long's Random Fourier Features PyTorch, which is typically under a permissive license. Compatibility for commercial use is not stated.

Limitations & Caveats

The repository is marked as "Repo Under Construction," suggesting ongoing development and potential for changes. Specific limitations or known issues are not detailed.