672 research outputs found
Shape-centered Representation Learning for Visible-Infrared Person Re-identification
Current Visible-Infrared Person Re-Identification (VI-ReID) methods
prioritize extracting distinguishing appearance features, ignoring the natural
resistance of body shape against modality changes. Initially, we gauged the
discriminative potential of shapes by a straightforward concatenation of shape
and appearance features. However, two unresolved issues persist in the
utilization of shape features. One pertains to the dependence on auxiliary
models for shape feature extraction in the inference phase, along with the
errors in generated infrared shapes due to the intrinsic modality disparity.
The other issue involves the inadequately explored correlation between shape
and appearance features. To tackle the aforementioned challenges, we propose
the Shape-centered Representation Learning framework (ScRL), which focuses on
learning shape features and appearance features associated with shapes.
Specifically, we devise the Shape Feature Propagation (SFP), facilitating
direct extraction of shape features from original images with minimal
complexity costs during inference. To restitute inaccuracies in infrared body
shapes at the feature level, we present the Infrared Shape Restitution (ISR).
Furthermore, to acquire appearance features related to shape, we design the
Appearance Feature Enhancement (AFE), which accentuates identity-related
features while suppressing identity-unrelated features guided by shape
features. Extensive experiments are conducted to validate the effectiveness of
the proposed ScRL. Achieving remarkable results, the Rank-1 (mAP) accuracy
attains 76.1%, 71.2%, 92.4% (72.6%, 52.9%, 86.7%) on the SYSU-MM01, HITSZ-VCM,
RegDB datasets respectively, outperforming existing state-of-the-art methods
Learning Cross-modality Information Bottleneck Representation for Heterogeneous Person Re-Identification
Visible-Infrared person re-identification (VI-ReID) is an important and
challenging task in intelligent video surveillance. Existing methods mainly
focus on learning a shared feature space to reduce the modality discrepancy
between visible and infrared modalities, which still leave two problems
underexplored: information redundancy and modality complementarity. To this
end, properly eliminating the identity-irrelevant information as well as making
up for the modality-specific information are critical and remains a challenging
endeavor. To tackle the above problems, we present a novel mutual information
and modality consensus network, namely CMInfoNet, to extract modality-invariant
identity features with the most representative information and reduce the
redundancies. The key insight of our method is to find an optimal
representation to capture more identity-relevant information and compress the
irrelevant parts by optimizing a mutual information bottleneck trade-off.
Besides, we propose an automatically search strategy to find the most prominent
parts that identify the pedestrians. To eliminate the cross- and intra-modality
variations, we also devise a modality consensus module to align the visible and
infrared modalities for task-specific guidance. Moreover, the global-local
feature representations can also be acquired for key parts discrimination.
Experimental results on four benchmarks, i.e., SYSU-MM01, RegDB,
Occluded-DukeMTMC, Occluded-REID, Partial-REID and Partial\_iLIDS dataset, have
demonstrated the effectiveness of CMInfoNet
Dual Gaussian-based Variational Subspace Disentanglement for Visible-Infrared Person Re-Identification
Visible-infrared person re-identification (VI-ReID) is a challenging and
essential task in night-time intelligent surveillance systems. Except for the
intra-modality variance that RGB-RGB person re-identification mainly overcomes,
VI-ReID suffers from additional inter-modality variance caused by the inherent
heterogeneous gap. To solve the problem, we present a carefully designed dual
Gaussian-based variational auto-encoder (DG-VAE), which disentangles an
identity-discriminable and an identity-ambiguous cross-modality feature
subspace, following a mixture-of-Gaussians (MoG) prior and a standard Gaussian
distribution prior, respectively. Disentangling cross-modality
identity-discriminable features leads to more robust retrieval for VI-ReID. To
achieve efficient optimization like conventional VAE, we theoretically derive
two variational inference terms for the MoG prior under the supervised setting,
which not only restricts the identity-discriminable subspace so that the model
explicitly handles the cross-modality intra-identity variance, but also enables
the MoG distribution to avoid posterior collapse. Furthermore, we propose a
triplet swap reconstruction (TSR) strategy to promote the above disentangling
process. Extensive experiments demonstrate that our method outperforms
state-of-the-art methods on two VI-ReID datasets.Comment: Accepted by ACM MM 2020 poster. 12 pages, 10 appendixe
Efficient Bilateral Cross-Modality Cluster Matching for Unsupervised Visible-Infrared Person ReID
Unsupervised visible-infrared person re-identification (USL-VI-ReID) aims to
match pedestrian images of the same identity from different modalities without
annotations. Existing works mainly focus on alleviating the modality gap by
aligning instance-level features of the unlabeled samples. However, the
relationships between cross-modality clusters are not well explored. To this
end, we propose a novel bilateral cluster matching-based learning framework to
reduce the modality gap by matching cross-modality clusters. Specifically, we
design a Many-to-many Bilateral Cross-Modality Cluster Matching (MBCCM)
algorithm through optimizing the maximum matching problem in a bipartite graph.
Then, the matched pairwise clusters utilize shared visible and infrared
pseudo-labels during the model training. Under such a supervisory signal, a
Modality-Specific and Modality-Agnostic (MSMA) contrastive learning framework
is proposed to align features jointly at a cluster-level. Meanwhile, the
cross-modality Consistency Constraint (CC) is proposed to explicitly reduce the
large modality discrepancy. Extensive experiments on the public SYSU-MM01 and
RegDB datasets demonstrate the effectiveness of the proposed method, surpassing
state-of-the-art approaches by a large margin of 8.76% mAP on average
Visible-Infrared Person Re-Identification via Patch-Mixed Cross-Modality Learning
Visible-infrared person re-identification (VI-ReID) aims to retrieve images
of the same pedestrian from different modalities, where the challenges lie in
the significant modality discrepancy. To alleviate the modality gap, recent
methods generate intermediate images by GANs, grayscaling, or mixup strategies.
However, these methods could ntroduce extra noise, and the semantic
correspondence between the two modalities is not well learned. In this paper,
we propose a Patch-Mixed Cross-Modality framework (PMCM), where two images of
the same person from two modalities are split into patches and stitched into a
new one for model learning. In this way, the modellearns to recognize a person
through patches of different styles, and the modality semantic correspondence
is directly embodied. With the flexible image generation strategy, the
patch-mixed images freely adjust the ratio of different modality patches, which
could further alleviate the modality imbalance problem. In addition, the
relationship between identity centers among modalities is explored to further
reduce the modality variance, and the global-to-part constraint is introduced
to regularize representation learning of part features. On two VI-ReID
datasets, we report new state-of-the-art performance with the proposed method.Comment: IJCAI2
Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification
RGB-Infrared (IR) person re-identification is very challenging due to the
large cross-modality variations between RGB and IR images. The key solution is
to learn aligned features to the bridge RGB and IR modalities. However, due to
the lack of correspondence labels between every pair of RGB and IR images, most
methods try to alleviate the variations with set-level alignment by reducing
the distance between the entire RGB and IR sets. However, this set-level
alignment may lead to misalignment of some instances, which limits the
performance for RGB-IR Re-ID. Different from existing methods, in this paper,
we propose to generate cross-modality paired-images and perform both global
set-level and fine-grained instance-level alignments. Our proposed method
enjoys several merits. First, our method can perform set-level alignment by
disentangling modality-specific and modality-invariant features. Compared with
conventional methods, ours can explicitly remove the modality-specific features
and the modality variation can be better reduced. Second, given cross-modality
unpaired-images of a person, our method can generate cross-modality paired
images from exchanged images. With them, we can directly perform instance-level
alignment by minimizing distances of every pair of images. Extensive
experimental results on two standard benchmarks demonstrate that the proposed
model favourably against state-of-the-art methods. Especially, on SYSU-MM01
dataset, our model can achieve a gain of 9.2% and 7.7% in terms of Rank-1 and
mAP. Code is available at https://github.com/wangguanan/JSIA-ReID.Comment: accepted by AAAI'2
- …