429 research outputs found
Image-Specific Information Suppression and Implicit Local Alignment for Text-based Person Search
Text-based person search (TBPS) is a challenging task that aims to search
pedestrian images with the same identity from an image gallery given a query
text. In recent years, TBPS has made remarkable progress and state-of-the-art
methods achieve superior performance by learning local fine-grained
correspondence between images and texts. However, most existing methods rely on
explicitly generated local parts to model fine-grained correspondence between
modalities, which is unreliable due to the lack of contextual information or
the potential introduction of noise. Moreover, existing methods seldom consider
the information inequality problem between modalities caused by image-specific
information. To address these limitations, we propose an efficient joint
Multi-level Alignment Network (MANet) for TBPS, which can learn aligned
image/text feature representations between modalities at multiple levels, and
realize fast and effective person search. Specifically, we first design an
image-specific information suppression module, which suppresses image
background and environmental factors by relation-guided localization and
channel attention filtration respectively. This module effectively alleviates
the information inequality problem and realizes the alignment of information
volume between images and texts. Secondly, we propose an implicit local
alignment module to adaptively aggregate all pixel/word features of image/text
to a set of modality-shared semantic topic centers and implicitly learn the
local fine-grained correspondence between modalities without additional
supervision and cross-modal interactions. And a global alignment is introduced
as a supplement to the local perspective. The cooperation of global and local
alignment modules enables better semantic alignment between modalities.
Extensive experiments on multiple databases demonstrate the effectiveness and
superiority of our MANet
Exploiting Cross Domain Relationships for Target Recognition
Cross domain recognition extracts knowledge from one domain to recognize samples from another domain of interest. The key to solving problems under this umbrella is to find out the latent connections between different domains. In this dissertation, three different cross domain recognition problems are studied by exploiting the relationships between different domains explicitly according to the specific real problems.
First, the problem of cross view action recognition is studied. The same action might seem quite different when observed from different viewpoints. Thus, how to use the training samples from a given camera view and perform recognition in another new view is the key point. In this work, reconstructable paths between different views are built to mirror labeled actions from one source view into one another target view for learning an adaptable classifier. The path learning takes advantage of the joint dictionary learning techniques with exploiting hidden information in the seemingly useless samples, making the recognition performance robust and effective.
Second, the problem of person re-identification is studied, which tries to match pedestrian images in non-overlapping camera views based on appearance features. In this work, we propose to learn a random kernel forest to discriminatively assign a specific distance metric to each pair of local patches from the two images in matching. The forest is composed by multiple decision trees, which are designed to partition the overall space of local patch-pairs into substantial subspaces, where a simple but effective local metric kernel can be defined to minimize the distance of true matches.
Third, the problem of multi-event detection and recognition in smart grid is studied. The signal of multi-event might not be a straightforward combination of some single-event signals because of the correlation among devices. In this work, a concept of ``root-pattern\u27\u27 is proposed that can be extracted from a collection of single-event signals, but also transferable to analyse the constituent components of multi-cascading-event signals based on an over-complete dictionary, which is designed according to the ``root-patterns\u27\u27 with temporal information subtly embedded.
The correctness and effectiveness of the proposed approaches have been evaluated by extensive experiments
Deep Heterogeneous Hashing for Face Video Retrieval
Retrieving videos of a particular person with face image as a query via
hashing technique has many important applications. While face images are
typically represented as vectors in Euclidean space, characterizing face videos
with some robust set modeling techniques (e.g. covariance matrices as exploited
in this study, which reside on Riemannian manifold), has recently shown
appealing advantages. This hence results in a thorny heterogeneous spaces
matching problem. Moreover, hashing with handcrafted features as done in many
existing works is clearly inadequate to achieve desirable performance for this
task. To address such problems, we present an end-to-end Deep Heterogeneous
Hashing (DHH) method that integrates three stages including image feature
learning, video modeling, and heterogeneous hashing in a single framework, to
learn unified binary codes for both face images and videos. To tackle the key
challenge of hashing on the manifold, a well-studied Riemannian kernel mapping
is employed to project data (i.e. covariance matrices) into Euclidean space and
thus enables to embed the two heterogeneous representations into a common
Hamming space, where both intra-space discriminability and inter-space
compatibility are considered. To perform network optimization, the gradient of
the kernel mapping is innovatively derived via structured matrix
backpropagation in a theoretically principled way. Experiments on three
challenging datasets show that our method achieves quite competitive
performance compared with existing hashing methods.Comment: 14 pages, 17 figures, 4 tables, accepted by IEEE Transactions on
Image Processing (TIP) 201
- …