54 research outputs found
A Survey on Soft Biometrics for Human Identification
The focus has been changed to multi-biometrics due to the security demands. The ancillary information extracted from primary biometric (face and body) traits such as facial measurements, gender, color of the skin, ethnicity, and height is called soft biometrics and can be integrated to improve the speed and overall system performance of a primary biometric system (e.g., fuse face with facial marks) or to generate human semantic interpretation description (qualitative) of a person and limit the search in the whole dataset when using gender and ethnicity (e.g., old African male with blue eyes) in a fusion framework. This chapter provides a holistic survey on soft biometrics that show major works while focusing on facial soft biometrics and discusses some of the features of extraction and classification techniques that have been proposed and show their strengths and limitations
Deep Learning Architectures for Heterogeneous Face Recognition
Face recognition has been one of the most challenging areas of research in biometrics and computer vision. Many face recognition algorithms are designed to address illumination and pose problems for visible face images. In recent years, there has been significant amount of research in Heterogeneous Face Recognition (HFR). The large modality gap between faces captured in different spectrum as well as lack of training data makes heterogeneous face recognition (HFR) quite a challenging problem. In this work, we present different deep learning frameworks to address the problem of matching non-visible face photos against a gallery of visible faces.
Algorithms for thermal-to-visible face recognition can be categorized as cross-spectrum feature-based methods, or cross-spectrum image synthesis methods. In cross-spectrum feature-based face recognition a thermal probe is matched against a gallery of visible faces corresponding to the real-world scenario, in a feature subspace. The second category synthesizes a visible-like image from a thermal image which can then be used by any commercial visible spectrum face recognition system. These methods also beneficial in the sense that the synthesized visible face image can be directly utilized by existing face recognition systems which operate only on the visible face imagery. Therefore, using this approach one can leverage the existing commercial-off-the-shelf (COTS) and government-off-the-shelf (GOTS) solutions. In addition, the synthesized images can be used by human examiners for different purposes.
There are some informative traits, such as age, gender, ethnicity, race, and hair color, which are not distinctive enough for the sake of recognition, but still can act as complementary information to other primary information, such as face and fingerprint. These traits, which are known as soft biometrics, can improve recognition algorithms while they are much cheaper and faster to acquire. They can be directly used in a unimodal system for some applications. Usually, soft biometric traits have been utilized jointly with hard biometrics (face photo) for different tasks in the sense that they are considered to be available both during the training and testing phases. In our approaches we look at this problem in a different way. We consider the case when soft biometric information does not exist during the testing phase, and our method can predict them directly in a multi-tasking paradigm.
There are situations in which training data might come equipped with additional information that can be modeled as an auxiliary view of the data, and that unfortunately is not available during testing. This is the LUPI scenario. We introduce a novel framework based on deep learning techniques that leverages the auxiliary view to improve the performance of recognition system. We do so by introducing a formulation that is general, in the sense that can be used with any visual classifier.
Every use of auxiliary information has been validated extensively using publicly available benchmark datasets, and several new state-of-the-art accuracy performance values have been set. Examples of application domains include visual object recognition from RGB images and from depth data, handwritten digit recognition, and gesture recognition from video.
We also design a novel aggregation framework which optimizes the landmark locations directly using only one image without requiring any extra prior which leads to robust alignment given arbitrary face deformations. Three different approaches are employed to generate the manipulated faces and two of them perform the manipulation via the adversarial attacks to fool a face recognizer. This step can decouple from our framework and potentially used to enhance other landmark detectors. Aggregation of the manipulated faces in different branches of proposed method leads to robust landmark detection.
Finally we focus on the generative adversarial networks which is a very powerful tool in synthesizing a visible-like images from the non-visible images. The main goal of a generative model is to approximate the true data distribution which is not known. In general, the choice for modeling the density function is challenging. Explicit models have the advantage of explicitly calculating the probability densities. There are two well-known implicit approaches, namely the Generative Adversarial Network (GAN) and Variational AutoEncoder (VAE) which try to model the data distribution implicitly. The VAEs try to maximize the data likelihood lower bound, while a GAN performs a minimax game between two players during its optimization. GANs overlook the explicit data density characteristics which leads to undesirable quantitative evaluations and mode collapse. This causes the generator to create similar looking images with poor diversity of samples. In the last chapter of thesis, we focus to address this issue in GANs framework
κ΄μ λ€μ€ 보νμ μΆμ μ μν κ³μΈ΅μ κΆ€μ λ§€μΉ κΈ°λ²
νμλ
Όλ¬Έ (λ°μ¬) -- μμΈλνκ΅ λνμ : 곡과λν μ κΈ°Β·μ»΄ν¨ν°κ³΅νλΆ, 2020. 8. μ΅μ§μ.The purpose of wide-area tracking problem is to track pedestrians that appear on cameras that overlap or do not overlap, regardless of the time interval or person density.
In a single camera tracking, data association using overlapping of the detection boxes is used to solve the tracking problem, but still has appearance ambiguity issues.
However, wide-area tracking requires a tracking scheme that focuses on the appearance similarity of humans, without the use of overlapping of detection boxes.
In this dissertation, we propose the tracking scheme for the Wide-area Multi-Pedestrian Tracking (WaMuPeT).
To achieve the WaMuPeT, we propose the trajectory matching in overlapping camera settings (Ch. 3), non-overlapping camera settings (Ch. 4) and robust trajectory matching in dense scene settings (Ch. 5).
In trajectory matching in overlapping camera settings (Ch. 3), we propose a novel deep-learning architecture for accurate 3-D localization and tracking of a pedestrian using multiple cameras.
The deep-learning network is composed of two networks: detection network and localization network.
The detection network yields the pedestrian detections and the localization network estimates the ground position of a pedestrian within its detection box.
In addition, an attentional pass filter is introduced to effectively connect the two networks.
Using the detection proposals and their 2-D grounding positions obtained from the two networks, multi-camera multi-target 3-D localization and tracking algorithm is developed through min-cost network flow approach.
In the experiments, it is shown that the proposed method improves the performance of 3-D localization and tracking.
In trajectory matching in non-overlapping camera settings (Ch. 4), we propose a novel re-ranking method using a ranking-reflected metric to measure the similarity between two ordered sets of -nearest neighbors (OKNN).
The proposed metric for ranking-reflected similarity (RSS) reflects the ranking of the shared elements between the two OKNNs.
Using RSS, a re-ranking procedure is proposed that prioritizes galleries having neighbors similar to a probe's neighbor in the perspective of ranking order.
In the experiment, we show that the proposed method improves the Re-ID accuracy by add-on to the state-of-the-art methods.
In robust trajectory matching in dense scene settings (Ch. 5), we propose a novel framework for multi-pedestrian tracking to generate robust trajectories in dense scene.
In the proposed tracking method, we propose the tracking method based on the trajectory matching by the strategy of divide and conquer method.
In this strategy, short-term, mid-term and long-term trajectories are generated by each trajectory merging stages, respectively.
Also we propose a novel deep-feature matching method called stable boundary selection (SBS).
In SBS matching, the detections are clustered by the group similarity of deep features, so that robust trajectories can be generated.
With the smoothing algorithms and the detection restoration algorithm, the proposed tracking method shows the state-of-the-art tracking accuracy in three public tracking dataset.κ΄μ μΆμ λ¬Έμ μ λͺ©μ μ μκ° κ°κ²©μ΄λ μ¬λ λ°λμ κ΄κ³μμ΄ κ²ΉμΉκ±°λ κ²ΉμΉμ§ μλ μΉ΄λ©λΌμ λνλλ 보νμλ₯Ό μΆμ νλ κ²μ΄λ€.
λ¨μΌ μΉ΄λ©λΌ μΆμ μμ κ°μ§ μμμ κ²ΉμΉ¨μ μ¬μ©νλ λ°μ΄ν° μ°κ²°μ μΆμ λ¬Έμ λ₯Ό ν΄κ²°νλ λ° μ¬μ©λμ§λ§ μ¬μ ν λͺ¨μ λͺ¨νΈμ± λ¬Έμ κ° μλ€.
κ·Έλ¬λ κ΄μ μΆμ μλ κ°μ§ μμμ κ²ΉμΉ¨μ μ¬μ©νμ§ μκ³ μ¬λμ μΈν μ μ¬μ±μ μ€μ μ λ μΆμ 체κ³κ° νμνλ€.
μ΄ λ
Όλ¬Έμμλ κ΄μ λ€μ€ 보νμ μΆμ (WaMuPeT)μ λν μΆμ 체κ³λ₯Ό μ μνλ€.
WaMuPeTλ₯Ό λ¬μ±νκΈ° μν΄ κ²ΉμΉλ μΉ΄λ©λΌ μ€μ (3 μ₯), κ²ΉμΉμ§ μλ μΉ΄λ©λΌ μ€μ (4 μ₯) μμμ κΆ€μ μΌμΉ κ·Έλ¦¬κ³ λΉ½λΉ½ν μ₯λ©΄ μ€μ (5 μ₯)μμ κ°μΈν κΆ€μ μΌμΉμ λν΄μ μ μνλ€.
κ²ΉμΉλ μΉ΄λ©λΌ μ€μ μμμ κΆ€μ λ§€μΉ (3 μ₯)μμλ μ¬λ¬ μΉ΄λ©λΌλ₯Ό μ¬μ©νμ¬ λ³΄νμλ₯Ό μ ννκ² 3D μ§μννκ³ μΆμ νκΈ°μν μλ‘μ΄ λ₯ λ¬λ μν€ν
μ²λ₯Ό μ μνλ€.
λ₯ λ¬λ λ€νΈμν¬λ κ°μ§ λ€νΈμν¬μ λ‘컬λΌμ΄μ μ΄μ
λ€νΈμν¬μ λ κ°μ§ λ€νΈμν¬λ‘ ꡬμ±λλ€.
νμ§ λ€νΈμν¬λ 보νμ νμ§λ₯Ό μ 곡νκ³ νμ§ν λ€νΈμν¬λ νμ§ μμ λ΄μμ 보νμμ μ§μ μμΉλ₯Ό μΆμ νλ€.
λν λ κ°μ λ€νΈμν¬λ₯Ό ν¨κ³Όμ μΌλ‘ μ°κ²°νκΈ° μν΄μ£Όμ ν¨μ€ νν°κ° λμ
λμλ€.
λ λ€νΈμν¬μμ μ»μ νμ§ μ μ λ° 2D μ μ§ μμΉλ₯Ό μ¬μ©νμ¬ μ΅μ λΉμ©μ λ€νΈμν¬ νλ¦ μ κ·Ό λ°©μμ ν΅ν΄ λ€μ€ μΉ΄λ©λΌ λ€μ€ λμ 3D μ§μν λ° μΆμ μκ³ λ¦¬μ¦μ΄ κ°λ°λλ€.
μ€νμμ μ μ λ λ°©λ²μ΄ 3D μ§μν λ° μΆμ μ±λ₯μ ν₯μμν€λ κ²μΌλ‘ λνλ¬λ€.
κ²ΉμΉμ§ μλ μΉ΄λ©λΌ μ€μ μμμ κΆ€μ μΌμΉ (4 μ₯)μμ, μ°λ¦¬λ μμκ° λ°μλ λ©νΈλ¦μ μ¬μ©νμ¬ λκ°μ μμκ° μ§μ λ -μ΅κ·Ό μ μ΄μ (OKNN) μΈνΈ μ¬μ΄μ μ μ¬μ±μ μΈ‘μ νλ€.
μμ λ°μ μ μ¬μ± (RSS)μ λν΄ μ μ λ λ©νΈλ¦μ λ OKNN μ¬μ΄μ 곡μ μμμ μμλ₯Ό λ°μν©λλ€.
RSSλ₯Ό μ¬μ©νμ¬, μμ μμμ κ΄μ μμ νλ‘λΈμ μ΄μκ³Ό μ μ¬ν μ΄μμ κ°λ κ°€λ¬λ¦¬λ₯Ό μ°μ μμ ννλ μ¬μμ μ μ°¨κ° μ μλλ€.
μ€νμμ μ μ λ λ°©λ²μ΄ μ΅μ λ°©λ²μ μΆκ°λμ΄ Re-ID μ νλκ° ν₯μλ¨μ 보μ¬μ€λ€.
κ³ λ°λ μ₯λ©΄ μ€μ μμ κ°λ ₯ν κΆ€μ μΌμΉ (5 μ₯)μμ, μ°λ¦¬λ κ³ λ°λ μ₯λ©΄μμ κ°λ ₯ν κΆ€μ μ μμ±νκΈ° μν΄ λ€μ€ 보νμ μΆμ μ μν μλ‘μ΄ νλ μ μν¬λ₯Ό μ μνλ€.
μ μλ μΆμ λ°©λ²μμλ λΆν λ° μ 볡 λ°©λ² μ λ΅μ λ°λ₯Έ κΆ€μ 맀μΉμ κΈ°λ°μΌλ‘ μΆμ λ°©λ²μ μ μνλ€.
μ΄ μ λ΅μμ, λ¨κΈ°, μ€κΈ° λ° μ₯κΈ° κΆ€μ μ κ°κ°μ κΆ€μ λ³ν© λ¨κ³μ μν΄ μμ±λλ€.
λν SBS (Stable Boundary Selection)λΌλ μλ‘μ΄ κΈ°λ₯ λ§€μΉ κΈ°λ²μ μ μνλ€.
SBS 맀μΉμμ, νμ§λ κΉμ νΉμ§μ κ·Έλ£Ή μ μ¬μ±μ μν΄ κ΅°μ§νλμ΄, κ°λ ₯ν κΆ€μ μ΄ μμ± λ μ μλ€.
μ μ λ μΆμ λ°©λ²μ νν μκ³ λ¦¬μ¦κ³Ό νμ§ λ³΅μ μκ³ λ¦¬μ¦μ ν΅ν΄ 3 κ°μ κ³΅κ° μΆμ λ°μ΄ν° μΈνΈμμ μ΅μ²¨λ¨ μΆμ μ νλλ₯Ό 보μ¬μ€λ€.Chapter 1 Introduction 1
1.1 Background 1
1.2 Related Works 4
1.2.1 Localization of Pedestrian Detection 4
1.2.2 Pedestrian Feature from Person Re-identification 5
1.2.3 Multi-Pedestrian Tracking 8
1.3 Contributions 8
1.4 Thesis Organization 10
Chapter 2 Problem Statements 11
2.1 Trajectory Matching in Overlapping Camera Settings 11
2.1.1 Challenges 11
2.1.2 Approach for the challenges 13
2.2 Trajectory Matching in Non-Overlapping Camera Settings 13
2.2.1 Challenges 13
2.2.2 Approach for the challenges 14
2.3 Robust Trajectory Matching in Dense Scene Settings 16
2.3.1 Challenges 16
2.3.2 Approach for the challenges 18
Chapter 3 Trajectory Matching in Overlapping Camera Settings 19
3.1 Overall Scheme 19
3.2 Network Design 20
3.3 MCMTT with Proposed Network 22
Chapter 4 Trajectory Matching in Non-overlapping Camera Settings 25
4.1 Overall Scheme 25
4.2 Proposed Method 30
4.2.1 Proposed Similarity Metric 30
4.2.2 Selection of A 31
4.2.3 Re-ranking Procedure 32
Chapter 5 Robust Trajectory Matching in Dense Scene Settings 35
5.1 Overall Scheme 35
5.2 Similarity Matrix Generation 39
5.3 Stable Boundary Selection 40
5.4 Trajectory Smoothing 42
5.5 Detection Restoration 46
5.6 Trajectory Merging Process 48
Chapter 6 Experiments 51
6.1 Dataset and Evaluation Metric 51
6.1.1 Trajectory Matching in Overlapping Camera Settings 51
6.1.2 Trajectory Matching in Non-overlapping Camera Settings 52
6.1.3 Robust Trajectory Matching in Dense Scene Settings 53
6.2 Results and Discussion 56
6.2.1 Trajectory Matching in Overlapping Camera Settings 56
6.2.2 Trajectory Matching in Non-overlapping Camera Settings 56
6.2.3 Robust Trajectory Matching in Dense Scene Settings 62
Chapter 7 Conclusions and Future Works 81
7.1 Concluding Remarks 81
7.2 Future Works 83
Abstract 97Docto
Person Re-identification: Past, Present and Future
Person re-identification (re-ID) has become increasingly popular in the community due to its application and research significance. It aims at spotting a person of interest in other cameras. In the early days, hand-crafted algorithms and small-scale evaluation were predominantly reported. Recent years have witnessed the emergence of large-scale datasets and deep learning systems which make use of large data volumes. Considering different tasks, we classify most current re-ID methods into two classes, i.e., image-based and video-based; in both tasks, hand-crafted and deep learning systems will be reviewed. Moreover, two new re-ID tasks which are much closer to real-world applications are described and discussed, i.e., end-to-end re-ID and fast re-ID in very large galleries. This paper: 1) introduces the history of person re-ID and its relationship with image classification and instance retrieval; 2) surveys a broad selection of the hand-crafted systems and the large-scale methods in both image- and video-based re-ID; 3) describes critical future directions in end-to-end re-ID and fast retrieval in large galleries; and 4) finally briefs some important yet under-developed issues
- β¦