171 research outputs found
Discrete Multi-modal Hashing with Canonical Views for Robust Mobile Landmark Search
Mobile landmark search (MLS) recently receives increasing attention for its
great practical values. However, it still remains unsolved due to two important
challenges. One is high bandwidth consumption of query transmission, and the
other is the huge visual variations of query images sent from mobile devices.
In this paper, we propose a novel hashing scheme, named as canonical view based
discrete multi-modal hashing (CV-DMH), to handle these problems via a novel
three-stage learning procedure. First, a submodular function is designed to
measure visual representativeness and redundancy of a view set. With it,
canonical views, which capture key visual appearances of landmark with limited
redundancy, are efficiently discovered with an iterative mining strategy.
Second, multi-modal sparse coding is applied to transform visual features from
multiple modalities into an intermediate representation. It can robustly and
adaptively characterize visual contents of varied landmark images with certain
canonical views. Finally, compact binary codes are learned on intermediate
representation within a tailored discrete binary embedding model which
preserves visual relations of images measured with canonical views and removes
the involved noises. In this part, we develop a new augmented Lagrangian
multiplier (ALM) based optimization method to directly solve the discrete
binary codes. We can not only explicitly deal with the discrete constraint, but
also consider the bit-uncorrelated constraint and balance constraint together.
Experiments on real world landmark datasets demonstrate the superior
performance of CV-DMH over several state-of-the-art methods
Deep Metric Multi-View Hashing for Multimedia Retrieval
Learning the hash representation of multi-view heterogeneous data is an
important task in multimedia retrieval. However, existing methods fail to
effectively fuse the multi-view features and utilize the metric information
provided by the dissimilar samples, leading to limited retrieval precision.
Current methods utilize weighted sum or concatenation to fuse the multi-view
features. We argue that these fusion methods cannot capture the interaction
among different views. Furthermore, these methods ignored the information
provided by the dissimilar samples. We propose a novel deep metric multi-view
hashing (DMMVH) method to address the mentioned problems. Extensive empirical
evidence is presented to show that gate-based fusion is better than typical
methods. We introduce deep metric learning to the multi-view hashing problems,
which can utilize metric information of dissimilar samples. On the
MIR-Flickr25K, MS COCO, and NUS-WIDE, our method outperforms the current
state-of-the-art methods by a large margin (up to 15.28 mean Average Precision
(mAP) improvement).Comment: Accepted by IEEE ICME 202
- …