3,348 research outputs found
Semantic Cross-View Matching
Matching cross-view images is challenging because the appearance and
viewpoints are significantly different. While low-level features based on
gradient orientations or filter responses can drastically vary with such
changes in viewpoint, semantic information of images however shows an invariant
characteristic in this respect. Consequently, semantically labeled regions can
be used for performing cross-view matching. In this paper, we therefore explore
this idea and propose an automatic method for detecting and representing the
semantic information of an RGB image with the goal of performing cross-view
matching with a (non-RGB) geographic information system (GIS). A segmented
image forms the input to our system with segments assigned to semantic concepts
such as traffic signs, lakes, roads, foliage, etc. We design a descriptor to
robustly capture both, the presence of semantic concepts and the spatial layout
of those segments. Pairwise distances between the descriptors extracted from
the GIS map and the query image are then used to generate a shortlist of the
most promising locations with similar semantic concepts in a consistent spatial
layout. An experimental evaluation with challenging query images and a large
urban area shows promising results
Learning Mutual Modulation for Self-Supervised Cross-Modal Super-Resolution
Self-supervised cross-modal super-resolution (SR) can overcome the difficulty
of acquiring paired training data, but is challenging because only
low-resolution (LR) source and high-resolution (HR) guide images from different
modalities are available. Existing methods utilize pseudo or weak supervision
in LR space and thus deliver results that are blurry or not faithful to the
source modality. To address this issue, we present a mutual modulation SR
(MMSR) model, which tackles the task by a mutual modulation strategy, including
a source-to-guide modulation and a guide-to-source modulation. In these
modulations, we develop cross-domain adaptive filters to fully exploit
cross-modal spatial dependency and help induce the source to emulate the
resolution of the guide and induce the guide to mimic the modality
characteristics of the source. Moreover, we adopt a cycle consistency
constraint to train MMSR in a fully self-supervised manner. Experiments on
various tasks demonstrate the state-of-the-art performance of our MMSR.Comment: ECCV 202
- …