1 research outputs found
GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization
Worldwide Geo-localization aims to pinpoint the precise location of images
taken anywhere on Earth. This task has considerable challenges due to immense
variation in geographic landscapes. The image-to-image retrieval-based
approaches fail to solve this problem on a global scale as it is not feasible
to construct a large gallery of images covering the entire world. Instead,
existing approaches divide the globe into discrete geographic cells,
transforming the problem into a classification task. However, their performance
is limited by the predefined classes and often results in inaccurate
localizations when an image's location significantly deviates from its class
center. To overcome these limitations, we propose GeoCLIP, a novel
CLIP-inspired Image-to-GPS retrieval approach that enforces alignment between
the image and its corresponding GPS locations. GeoCLIP's location encoder
models the Earth as a continuous function by employing positional encoding
through random Fourier features and constructing a hierarchical representation
that captures information at varying resolutions to yield a semantically rich
high-dimensional feature suitable to use even beyond geo-localization. To the
best of our knowledge, this is the first work employing GPS encoding for
geo-localization. We demonstrate the efficacy of our method via extensive
experiments and ablations on benchmark datasets. We achieve competitive
performance with just 20% of training data, highlighting its effectiveness even
in limited-data settings. Furthermore, we qualitatively demonstrate
geo-localization using a text query by leveraging CLIP backbone of our image
encoder. The project webpage is available at:
https://vicentevivan.github.io/GeoCLIPComment: Accepted at NeurIPS 202