2 research outputs found
Effective Image Retrieval via Multilinear Multi-index Fusion
Multi-index fusion has demonstrated impressive performances in retrieval task
by integrating different visual representations in a unified framework.
However, previous works mainly consider propagating similarities via neighbor
structure, ignoring the high order information among different visual
representations. In this paper, we propose a new multi-index fusion scheme for
image retrieval. By formulating this procedure as a multilinear based
optimization problem, the complementary information hidden in different indexes
can be explored more thoroughly. Specially, we first build our multiple indexes
from various visual representations. Then a so-called index-specific functional
matrix, which aims to propagate similarities, is introduced for updating the
original index. The functional matrices are then optimized in a unified tensor
space to achieve a refinement, such that the relevant images can be pushed more
closer. The optimization problem can be efficiently solved by the augmented
Lagrangian method with theoretical convergence guarantee. Unlike the
traditional multi-index fusion scheme, our approach embeds the multi-index
subspace structure into the new indexes with sparse constraint, thus it has
little additional memory consumption in online query stage. Experimental
evaluation on three benchmark datasets reveals that the proposed approach
achieves the state-of-the-art performance, i.e., N-score 3.94 on UKBench, mAP
94.1\% on Holiday and 62.39\% on Market-1501.Comment: 12 page
Cross-Spectrum Dual-Subspace Pairing for RGB-infrared Cross-Modality Person Re-Identification
Due to its potential wide applications in video surveillance and other
computer vision tasks like tracking, person re-identification (ReID) has become
popular and been widely investigated. However, conventional person
re-identification can only handle RGB color images, which will fail at dark
conditions. Thus RGB-infrared ReID (also known as Infrared-Visible ReID or
Visible-Thermal ReID) is proposed. Apart from appearance discrepancy in
traditional ReID caused by illumination, pose variations and viewpoint changes,
modality discrepancy produced by cameras of the different spectrum also exists,
which makes RGB-infrared ReID more difficult. To address this problem, we focus
on extracting the shared cross-spectrum features of different modalities. In
this paper, a novel multi-spectrum image generation method is proposed and the
generated samples are utilized to help the network to find discriminative
information for re-identifying the same person across modalities. Another
challenge of RGB-infrared ReID is that the intra-person (images from the same
person) discrepancy is often larger than the inter-person (images from
different persons) discrepancy, so a dual-subspace pairing strategy is proposed
to alleviate this problem. Combining those two parts together, we also design a
one-stream neural network combining the aforementioned methods to extract
compact representations of person images, called Cross-spectrum Dual-subspace
Pairing (CDP) model. Furthermore, during the training process, we also propose
a Dynamic Hard Spectrum Mining method to automatically mine more hard samples
from hard spectrum based on the current model state to further boost the
performance. Extensive experimental results on two public datasets, SYSU-MM01
with RGB + near-infrared images and RegDB with RGB + far-infrared images, have
demonstrated the efficiency and generality of our proposed method