Learning Discriminative Features for Person Re-Identification

For fulfilling the requirements of public safety in modern cities, more and more large-scale surveillance camera systems are deployed, resulting in an enormous amount of visual data. Automatically processing and interpreting these data promote the development and application of visual data analytic technologies. As one of the important research topics in surveillance systems, person re-identification (re-id) aims at retrieving the target person across non-overlapping camera-views that are implemented in a number of distributed space-time locations. It is a fundamental problem for many practical surveillance applications, eg, person search, cross-camera tracking, multi-camera human behavior analysis and prediction, and it received considerable attentions nowadays from both academic and industrial domains. Learning discriminative feature representation is an essential task in person re-id. Although many methodologies have been proposed, discriminative re-id feature extraction is still a challenging problem due to: (1) Intra- and inter-personal variations. The intrinsic properties of the camera deployment in surveillance system lead to various changes in person poses, view-points, illumination conditions etc. This may result in the large intra-personal variations and/or small inter-personal variations, thus incurring problems in matching person images. (2) Domain variations. The domain variations between different datasets give rise to the problem of generalization capability of re-id model. Directly applying a re-id model trained on one dataset to another one usually causes a large performance degradation. (3) Difficulties in data creation and annotation. Existing person re-id methods, especially deep re-id methods, rely mostly on a large set of inter-camera identity labelled training data, requiring a tedious data collection and annotation process. This leads to poor scalability in practical person re-id applications. Corresponding to the challenges in learning discriminative re-id features, this thesis contributes to the re-id domain by proposing three related methodologies and one new re-id setting: (1) Gaussian mixture importance estimation. Handcrafted features are usually not discriminative enough for person re-id because of noisy information, such as background clutters. To precisely evaluate the similarities between person images, the main task of distance metric learning is to filter out the noisy information. Keep It Simple and Straightforward MEtric (KISSME) is an effective method in person re-id. However, it is sensitive to the feature dimensionality and cannot capture the multi-modes in dataset. To this end, a Gaussian Mixture Importance Estimation re-id approach is proposed, which exploits the Gaussian Mixture Models for estimating the observed commonalities of similar and dissimilar person pairs in the feature space. (2) Unsupervised domain-adaptive person re-id based on pedestrian attributes. In person re-id, person identities are usually not overlapped among different domains (or datasets) and this raises the difficulties in generalizing re-id models. Different from person identity, pedestrian attributes, eg., hair length, clothes type and color, are consistent across different domains (or datasets). However, most of re-id datasets lack attribute annotations. On the other hand, in the field of pedestrian attribute recognition, there is a number of datasets labeled with attributes. Exploiting such data for re-id purpose can alleviate the shortage of attribute annotations in re-id domain and improve the generalization capability of re-id model. To this end, an unsupervised domain-adaptive re-id feature learning framework is proposed to make full use of attribute annotations. Specifically, an existing unsupervised domain adaptation method has been extended to transfer attribute-based features from attribute recognition domain to the re-id domain. With the proposed re-id feature learning framework, the domain invariant feature representations can be effectively extracted. (3) Intra-camera supervised person re-id. Annotating the large-scale re-id datasets requires a tedious data collection and annotation process and therefore leads to poor scalability in practical person re-id applications. To overcome this fundamental limitation, a new person re-id setting is considered without inter-camera identity association but only with identity labels independently annotated within each camera-view. This eliminates the most time-consuming and tedious inter-camera identity association annotating process and thus significantly reduces the amount of human efforts required during annotation. It hence gives rise to a more scalable and more feasible learning scenario, which is named as Intra-Camera Supervised (ICS) person re-id. Under this ICS setting, a new re-id method, i.e., Multi-task Mulit-label (MATE) learning method, is formulated. Given no inter-camera association, MATE is specially designed for self-discovering the inter-camera identity correspondence. This is achieved by inter-camera multi-label learning under a joint multi-task inference framework. In addition, MATE can also efficiently learn the discriminative re-id feature representations using the available identity labels within each camera-view

Person Re-identification by Local Maximal Occurrence Representation and Metric Learning

Person re-identification is an important technique towards automatic search of a person's presence in a surveillance video. Two fundamental problems are critical for person re-identification, feature representation and metric learning. An effective feature representation should be robust to illumination and viewpoint changes, and a discriminant metric should be learned to match various person images. In this paper, we propose an effective feature representation called Local Maximal Occurrence (LOMO), and a subspace and metric learning method called Cross-view Quadratic Discriminant Analysis (XQDA). The LOMO feature analyzes the horizontal occurrence of local features, and maximizes the occurrence to make a stable representation against viewpoint changes. Besides, to handle illumination variations, we apply the Retinex transform and a scale invariant texture operator. To learn a discriminant metric, we propose to learn a discriminant low dimensional subspace by cross-view quadratic discriminant analysis, and simultaneously, a QDA metric is learned on the derived subspace. We also present a practical computation method for XQDA, as well as its regularization. Experiments on four challenging person re-identification databases, VIPeR, QMUL GRID, CUHK Campus, and CUHK03, show that the proposed method improves the state-of-the-art rank-1 identification rates by 2.2%, 4.88%, 28.91%, and 31.55% on the four databases, respectively.Comment: This paper has been accepted by CVPR 2015. For source codes and extracted features please visit http://www.cbsr.ia.ac.cn/users/scliao/projects/lomo_xqda

Using latent features for short-term person re-identification with RGB-D cameras

This paper presents a system for people re-identification in uncontrolled scenarios using RGB-depth cameras. Compared to conventional RGB cameras, the use of depth information greatly simplifies the tasks of segmentation and tracking. In a previous work, we proposed a similar architecture where people were characterized using color-based descriptors that we named bodyprints. In this work, we propose the use of latent feature models to extract more relevant information from the bodyprint descriptors by reducing their dimensionality. Latent features can also cope with missing data in case of occlusions. Different probabilistic latent feature models, such as probabilistic principal component analysis and factor analysis, are compared in the paper. The main difference between the models is how the observation noise is handled in each case. Re-identification experiments have been conducted in a real store where people behaved naturally. The results show that the use of the latent features significantly improves the re-identification rates compared to state-of-the-art works.The work presented in this paper has been funded by the Spanish Ministry of Science and Technology under the CICYT contract TEVISMART, TEC2009-09146.Oliver Moll, J.; Albiol Colomer, A.; Albiol Colomer, AJ.; Mossi García, JM. (2016). Using latent features for short-term person re-identification with RGB-D cameras. Pattern Analysis and Applications. 19(2):549-561. https://doi.org/10.1007/s10044-015-0489-8S549561192http://kinectforwindows.org/http://www.gpiv.upv.es/videoresearch/personindexing.htmlAlbiol A, Albiol A, Oliver J, Mossi JM (2012) Who is who at different cameras. Matching people using depth cameras. Comput Vis IET 6(5):378–387Bak S, Corvee E, Bremond F, Thonnat M (2010) Person re-identification using haar-based and dcd-based signature. In: 2nd workshop on activity monitoring by multi-camera surveillance systems, AMMCSS 2010, in conjunction with 7th IEEE international conference on advanced video and signal-based surveillance, AVSS. AVSSBak S, Corvee E, Bremond F, Thonnat M (2010) Person re-identification using spatial covariance regions of human body parts. In: Seventh IEEE international conference on advanced video and signal based surveillance. pp. 435–440Bak S, Corvee E, Bremond F, Thonnat M (2011) Multiple-shot human re-identification by mean riemannian covariance grid. In: Advanced video and signal-based surveillance. Klagenfurt, Autriche. http://hal.inria.fr/inria-00620496Baltieri D, Vezzani R, Cucchiara R, Utasi A, BenedeK C, Szirányi T (2011) Multi-view people surveillance using 3d information. In: ICCV workshops. pp. 1817–1824Barbosa BI, Cristani M, Del Bue A, Bazzani L, Murino V (2012) Re-identification with rgb-d sensors. In: First international workshop on re-identificationBasilevsky A (1994) Statistical factor analysis and related methods: theory and applications. Willey, New YorkBäuml M, Bernardin K, Fischer k, Ekenel HK, Stiefelhagen R (2010) Multi-pose face recognition for person retrieval in camera networks. In: International conference on advanced video and signal-based surveillanceBazzani L, Cristani M, Perina A, Farenzena M, Murino V (2010) Multiple-shot person re-identification by hpe signature. In: Proceedings of the 2010 20th international conference on pattern recognition. Washington, DC, USA, pp. 1413–1416Bird ND, Masoud O, Papanikolopoulos NP, Isaacs A (2005) Detection of loitering individuals in public transportation areas. IEEE Trans Intell Transp Syst 6(2):167–177Bishop CM (2006) Pattern recognition and machine learning (information science and statistics). Springer, SecaucusCha SH (2007) Comprehensive survey on distance/similarity measures between probability density functions. Int J Math Models Methods Appl Sci 1(4):300–307Cheng YM, Zhou WT, Wang Y, Zhao CH, Zhang SW (2009) Multi-camera-based object handoff using decision-level fusion. In: Conference on image and signal processing. pp. 1–5Dikmen M, Akbas E, Huang TS, Ahuja N (2010) Pedestrian recognition with a learned metric. In: Asian conference in computer visionDoretto G, Sebastian T, Tu P, Rittscher J (2011) Appearance-based person reidentification in camera networks: problem overview and current approaches. J Ambient Intell Humaniz Comput 2:1–25Farenzena M, Bazzani L, Perina A, Murino V, Cristani M (2010) Person re-identification by symmetry-driven accumulation of local features. In: Proceedings of the 2010 IEEE computer society conference on computer vision and pattern recognition (CVPR 2010). IEEE Computer Society, San Francisco, CA, USAFodor I (2002) A survey of dimension reduction techniques. Technical report. Lawrence Livermore National LaboratoryFreund Y, Iyer R, Schapire RE, Singer Y (2003) An efficient boosting algorithm for combining preferences. J Mach Learn Res 4:933–969Gandhi T, Trivedi M (2006) Panoramic appearance map (pam) for multi-camera based person re-identification. Advanced Video and Signal Based Surveillance, IEEE Conference on, p. 78Garcia J, Gardel A, Bravo I, Lazaro J (2014) Multiple view oriented matching algorithm for people reidentification. Ind Inform IEEE Trans 10(3):1841–1851Gheissari N, Sebastian TB, Hartley R (2006) Person reidentification using spatiotemporal appearance. CVPR 2:1528–1535Gray D, Brennan S, Tao H (2007) Evaluating appearance models for recognition, reacquisition, and tracking. In: Proceedings of IEEE international workshop on performance evaluation for tracking and surveillance (PETS)Gray D, Tao H (2008) Viewpoint invariant pedestrian recognition with an ensemble of localized features. In: Proceedings of the 10th european conference on computer vision: part I. Berlin, pp. 262–275 (2008)Ilin A, Raiko T (2010) Practical approaches to principal component analysis in the presence of missing values. J Mach Learn Res 99:1957–2000Javed O, Shafique O, Rasheed Z, Shah M (2008) Modeling inter-camera space–time and appearance relationships for tracking across non-overlapping views. Comput Vis Image Underst 109(2):146–162Kai J, Bodensteiner C, Arens M (2011) Person re-identification in multi-camera networks. In: Computer vision and pattern recognition workshops (CVPRW), 2011 IEEE computer society conference on, pp. 55–61Kuo CH, Huang C, Nevatia R (2010) Inter-camera association of multi-target tracks by on-line learned appearance affinity models. Proceedings of the 11th european conference on computer vision: part I, ECCV’10. Springer, Berlin, pp 383–396Lan R, Zhou Y, Tang YY, Chen C (2014) Person reidentification using quaternionic local binary pattern. In: Multimedia and expo (ICME), 2014 IEEE international conference on, pp. 1–6Loy CC, Liu C, Gong S (2013) Person re-identification by manifold ranking. In: icip. pp. 3318–3325Madden C, Cheng E, Piccardi M (2007) Tracking people across disjoint camera views by an illumination-tolerant appearance representation. Mach Vis Appl 18:233–247Mazzon R, Tahir SF, Cavallaro A (2012) Person re-identification in crowd. Pattern Recogn Lett 33(14):1828–1837Oliveira IO, Souza Pio JL (2009) People reidentification in a camera network. In: Eighth IEEE international conference on dependable, autonomic and secure computing. pp. 461–466Papadakis P, Pratikakis I, Theoharis T, Perantonis SJ (2010) Panorama: a 3d shape descriptor based on panoramic views for unsupervised 3d object retrieval. Int J Comput Vis 89(2–3):177–192Prosser B, Zheng WS, Gong S, Xiang T (2010) Person re-identification by support vector ranking. In: Proceedings of the British machine vision conference. BMVA Press, pp. 21.1–21.11Roweis S (1998) Em algorithms for pca and spca. In: Advances in neural information processing systems. MIT Press, Cambridge, pp. 626–632 (1998)Pedagadi S, Orwell J, Velastin S, Boghossian B (2013) Local fisher discriminant analysis for pedestrian re-identification. In: CVPR. pp. 3318–3325Satta R, Fumera G, Roli F (2012) Fast person re-identification based on dissimilarity representations. Pattern Recogn Lett, Special Issue on Novel Pattern Recognition-Based Methods for Reidentification in Biometric Context 33:1838–1848Tao D, Jin L, Wang Y, Li X (2015) Person reidentification by minimum classification error-based kiss metric learning. Cybern IEEE Trans 45(2):242–252Tipping ME, Bishop CM (1999) Probabilistic principal component analysis. J R Stat Soc Ser B 61:611–622Tisse CL, Martin L, Torres L, Robert M (2002) Person identification technique using human iris recognition. In: Proceedings of vision interface, pp 294–299Vandergheynst P, Bierlaire M, Kunt M, Alahi A (2009) Cascade of descriptors to detect and track objects across any network of cameras. Comput Vis Image Underst, pp 1413–1416Verbeek J (2009) Notes on probabilistic pca with missing values. Technical reportWang D, Chen CO, Chen TY, Lee CT (2009) People recognition for entering and leaving a video surveillance area. In: Fourth international conference on innovative computing, information and control. pp. 334–337Zhang Z, Troje NF (2005) View-independent person identification from human gait. Neurocomputing 69:250–256Zhao T, Aggarwal M, Kumar R, Sawhney H (2005) Real-time wide area multi-camera stereo tracking. In: IEEE computer society conference on computer vision and pattern recognition. pp. 976–983Zheng S, Xie B, Huang K, Tao D (2011) Multi-view pedestrian recognition using shared dictionary learning with group sparsity. In: Lu BL, Zhang L, Kwok JT (eds) ICONIP (3), Lecture notes in computer science, vol 7064. Springer, New York, pp. 629–638Zheng WS, Gong S, Xiang T (2011) Person re-identification by probabilistic relative distance comparison. In: Computer vision and pattern recognition (CVPR), 2011 IEEE conference on. pp. 649–65

Beyond Intra-modality: A Survey of Heterogeneous Person Re-identification

An efficient and effective person re-identification (ReID) system relieves the users from painful and boring video watching and accelerates the process of video analysis. Recently, with the explosive demands of practical applications, a lot of research efforts have been dedicated to heterogeneous person re-identification (Hetero-ReID). In this paper, we provide a comprehensive review of state-of-the-art Hetero-ReID methods that address the challenge of inter-modality discrepancies. According to the application scenario, we classify the methods into four categories -- low-resolution, infrared, sketch, and text. We begin with an introduction of ReID, and make a comparison between Homogeneous ReID (Homo-ReID) and Hetero-ReID tasks. Then, we describe and compare existing datasets for performing evaluations, and survey the models that have been widely employed in Hetero-ReID. We also summarize and compare the representative approaches from two perspectives, i.e., the application scenario and the learning pipeline. We conclude by a discussion of some future research directions. Follow-up updates are avaible at: https://github.com/lightChaserX/Awesome-Hetero-reIDComment: Accepted by IJCAI 2020. Project url: https://github.com/lightChaserX/Awesome-Hetero-reI

Crossing Generative Adversarial Networks for Cross-View Person Re-identification

Person re-identification (\textit{re-id}) refers to matching pedestrians across disjoint yet non-overlapping camera views. The most effective way to match these pedestrians undertaking significant visual variations is to seek reliably invariant features that can describe the person of interest faithfully. Most of existing methods are presented in a supervised manner to produce discriminative features by relying on labeled paired images in correspondence. However, annotating pair-wise images is prohibitively expensive in labors, and thus not practical in large-scale networked cameras. Moreover, seeking comparable representations across camera views demands a flexible model to address the complex distributions of images. In this work, we study the co-occurrence statistic patterns between pairs of images, and propose to crossing Generative Adversarial Network (Cross-GAN) for learning a joint distribution for cross-image representations in a unsupervised manner. Given a pair of person images, the proposed model consists of the variational auto-encoder to encode the pair into respective latent variables, a proposed cross-view alignment to reduce the view disparity, and an adversarial layer to seek the joint distribution of latent representations. The learned latent representations are well-aligned to reflect the co-occurrence patterns of paired images. We empirically evaluate the proposed model against challenging datasets, and our results show the importance of joint invariant features in improving matching rates of person re-id with comparison to semi/unsupervised state-of-the-arts.Comment: 12 pages. arXiv admin note: text overlap with arXiv:1702.03431 by other author