Fisher Motion Descriptor for Multiview Gait Recognition

The goal of this paper is to identify individuals by analyzing their gait. Instead of using binary silhouettes as input data (as done in many previous works) we propose and evaluate the use of motion descriptors based on densely sampled short-term trajectories. We take advantage of state-of-the-art people detectors to de ne custom spatial con gurations of the descriptors around the target person, obtaining a rich representation of the gait motion. The local motion features (described by the Divergence-Curl-Shear descriptor [1]) extracted on the di erent spatial areas of the person are combined into a single high-level gait descriptor by using the Fisher Vector encoding [2]. The proposed approach, coined Pyramidal Fisher Motion, is experimentally validated on `CASIA' dataset [3] (parts B and C), `TUM GAID' dataset [4], `CMU MoBo' dataset [5] and the recent `AVA Multiview Gait' dataset [6]. The results show that this new approach achieves state-of-the-art results in the problem of gait recognition, allowing to recognize walking people from diverse viewpoints on single and multiple camera setups, wearing di erent clothes, carrying bags, walking at diverse speeds and not limited to straight walking paths

A multilevel paradigm for deep convolutional neural network features selection with an application to human gait recognition

Human gait recognition (HGR) shows high importance in the area of video surveillance due to remote access and security threats. HGR is a technique commonly used for the identification of human style in daily life. However, many typical situations like change of clothes condition and variation in view angles degrade the system performance. Lately, different machine learning (ML) techniques have been introduced for video surveillance which gives promising results among which deep learning (DL) shows best performance in complex scenarios. In this article, an integrated framework is proposed for HGR using deep neural network and fuzzy entropy controlled skewness (FEcS) approach. The proposed technique works in two phases: In the first phase, deep convolutional neural network (DCNN) features are extracted by pre-trained CNN models (VGG19 and AlexNet) and their information is mixed by parallel fusion approach. In the second phase, entropy and skewness vectors are calculated from fused feature vector (FV) to select best subsets of features by suggested FEcS approach. The best subsets of picked features are finally fed to multiple classifiers and finest one is chosen on the basis of accuracy value. The experiments were carried out on four well-known datasets, namely, AVAMVG gait, CASIA A, B and C. The achieved accuracy of each dataset was 99.8, 99.7, 93.3 and 92.2%, respectively. Therefore, the obtained overall recognition results lead to conclude that the proposed system is very promising

Gait recognition from multiple view-points

A la finalización de la tesis, la principal conclusión que se extrae es que la forma de andar permite identificar a las personas con una buena precisión (superior al 90 por ciento y llegando al 99 por ciento en determinados casos). Centrándonos en los diferentes enfoques desarrollados, el método basado en características extraídas a mano está especialmente indicado para bases de datos pequeñas en cuanto a número de muestras, ya que obtiene una buena precisión necesitando pocos datos de entrenamiento. Por otro lado, la aproximación basada en deep learning permite obtener buenos resultados para bases de datos grandes con la ventaja de que el tamaño de entrada puede ser muy pequeño, permitiendo una ejecución muy rápida. El enfoque incremental está especialmente indicado para entornos en los que se requieran añadir nuevos sujetos al sistema sin tener que entrenar el método de nuevo debido a los altos costes de tiempo y energía. Por último, el estudio de consumo nos ha permitido definir una serie de recomendaciones para poder minimizar el consumo de energía durante el entrenamiento de las redes profundas sin penalizar la precisión de las mismas. Fecha de lectura de Tesis Doctoral: 14 de diciembre 2018.Arquitectura de Computadores Resumen tesis: La identificación automática de personas está ganando mucha importancia en los últimos años ya que se puede aplicar en entornos que deben ser seguros (aeropuertos, centrales nucleares, etc) para agilizar todos los procesos de acceso. La mayoría de soluciones desarrolladas para este problema se basan en un amplio abanico de características físicas de los sujetos, como pueden ser el iris, la huella dactilar o la cara. Sin embargo, este tipo de técnicas tienen una serie de limitaciones ya que requieren la colaboración por parte del sujeto a identificar o bien son muy sensibles a cambios en la apariencia. Sin embargo, el reconocimiento del paso es una forma no invasiva de implementar estos controles de seguridad y, adicionalmente, no necesita la colaboración del sujeto. Además, es robusto frente a cambios en la apariencia del individuo ya que se centra en el movimiento. El objetivo principal de esta tesis es desarrollar un nuevo método para la identificación de personas a partir de la forma de caminar en entornos de múltiples vistas. Como entrada usamos el flujo óptico que proporciona una información muy rica sobre el movimiento del sujeto mientras camina. Para cumplir este objetivo, se han desarrollado dos técnicas diferentes: una basada en un enfoque tradicional de visión por computador donde se extraen manualmente características que definen al sujeto y, una segunda aproximación basada en aprendizaje profundo (deep learning) donde el propio método extrae sus características y las clasifica automáticamente. Además, para este último enfoque, se ha desarrollado una implementación basada en aprendizaje incremental para añadir nuevas clases sin entrenar el modelo desde cero y, un estudio energético para optimizar el consumo de energía durante el entrenamiento

An efficient gait recognition method for known and unknown covariate conditions

YesGait is a unique non-invasive biometric form that can be utilized to effectively recognize persons, even when they prove to be uncooperative. Computer-aided gait recognition systems usually use image sequences without considering covariates like clothing and possessions of carrier bags whilst on the move. Similarly, in gait recognition, there may exist unknown covariate conditions that may affect the training and testing conditions for a given individual. Consequently, common techniques for gait recognition and measurement require a degree of intervention leading to the introduction of unknown covariate conditions, and hence this significantly limits the practical use of the present gait recognition and analysis systems. To overcome these key issues, we propose a method of gait analysis accounting for both known and unknown covariate conditions. For this purpose, we propose two methods, i.e., a Convolutional Neural Network (CNN) based gait recognition and a discriminative features-based classification method for unknown covariate conditions. The first method can handle known covariate conditions efficiently while the second method focuses on identifying and selecting unique covariate invariant features from the gallery and probe sequences. The feature set utilized here includes Local Binary Patterns (LBP), Histogram of Oriented Gradients (HOG), and Haralick texture features. Furthermore, we utilize the Fisher Linear Discriminant Analysis for dimensionality reduction and selecting the most discriminant features. Three classifiers, namely Random Forest, Support Vector Machine (SVM), and Multilayer Perceptron are used for gait recognition under strict unknown covariate conditions. We evaluated our results using CASIA and OUR-ISIR datasets for both clothing and speed variations. As a result, we report that on average we obtain an accuracy of 90.32% for the CASIA dataset with unknown covariates and similarly performed excellently on the ISIR dataset. Therefore, our proposed method outperforms existing methods for gait recognition under known and unknown covariate conditions.This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (NRF-2019R1F1A1060668)

View-invariant gait person re-identification with spatial and temporal attention

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonPerson re-identification at a distance across multiple none overlapping cameras has been an active research area for years. In the past ten years, Short term Person Re-Id techniques have made great strides in terms of accuracy using only appearance features in limited environments. However, massive intraclass variations and inter-class confusion limit their ability to be used in practical applications. Moreover, appearance consistency can only be assumed in a short time span from one camera to the other. Since the holistic appearance will change drastically over days and weeks, the technique, as mentioned above, will be ineffective. Practical applications usually require a long-term solution in which the subject appearance and clothing might have changed after a significant period has elapsed. Facing these problems, soft biometric features such as Gait have been proposed in the past. Nevertheless, even Gait can vary with illness, ageing and changes in the emotional state, changes in walking surfaces, shoe type, clothes type, objects carried by the subject and even clutter in the scene. Therefore, Gait is considered a temporal cue that could provide biometric motion information. On the other hand, the shape of the human body could be viewed as a spatial signal which can produce valuable information. So, extracting discriminative features from both spatial and temporal domains would be very beneficial to this research. Therefore, this thesis focuses on finding the best and most robust method to tackle the gait human Re-identification problem and solve it for practical applications. In real-world surveillance scenarios, the human gait cycle is primarily abnormal. These abnormalities include but not limited to temporal and spatial characteristics changes such as walking speed, broken gait phase and most importantly, varied camera angles. Our work performed an extensive literature study on spatial and temporal gait feature extraction methods with a focus on deep learning. Next, we conducted a comparative study and proposed a spatial-temporal approach for gait feature extraction using the fusion of multiple modalities, including optical-flow, raw silhouettes and RGB images. This approach was tested on two of the most challenging publicly available datasets for gait recognition TUM-GAID and CASIA-B, with excellent results presented in chapter 3. Furthermore, a modern spatial-temporal attention mechanism was proposed and tested on CASIA-B and OULP datasets which learns salient features independent of the gait cycle and view variations. The spatial attention layer in the proposed method extracts the spatial feature maps using a two-layered architecture that are fused using late fusion. It can pay attention to the identity-related salient regions in silhouette sequences discriminatively using the spatial feature maps. The temporal attention layer consists of an LSTM that encodes the temporal motion for silhouette sequences. It uses the encoded output vectors in the temporal attention architecture to focus on the most critical timesteps in the gait cycle and discard the rest. Furthermore, we improved the performance of our method by mapping our extracted spatial-temporal gait features to a discriminative null space for use in our Siamese architecture for crossmatching. We also conducted an element removal experiment on each segment of our spatial-temporal attentional network to gain insight into each component’s contribution to the performance. Our method showed outstanding robustness against abnormal gait cycles as well as viewpoint variations on both benchmark datasets

Investigation of gait representations and partial body gait recognition

Recognising an individual by the way they walk is one of the most popular research subjects within the field of soft biometrics in last few decades. The advancement of technology and equipment such as Close Circuit Television (CCTV), wireless internet and wearable sensors makes it easier to obtain gait data than ever before. The gait biometric can be used widely and in different areas such as biomedical, forensic and surveillance. However, gait recognition still has many challenges and fundamental issues. All of these problems only serve as a researcher’s motivation to learn more about various gait topics to overcome the challenges and improve the field of gait recognition. Gait recognition currently has high performance when carried out under very specific conditions such as normal walking, obstruction from certain types of clothing and fixed camera view angles. When the aforementioned conditions are changed, the classification rate dramatically drops. This study aims to solve the problems of clothing, carrying objects and camera view angles within the indoor environment and video-based data collection. Two gait related databases used for testing in this study are CASIA dataset B and OU-ISIR Large population dataset with Bag (OU-LP-Bag). Three main tasks will be tested with CASIA dataset B while only gait recognition is tested with OU-LP-Bag. The gait recognition framework is developed to solve the three main tasks including gait recognition by identical view, view classification and cross view recognition. This framework uses gait images sequence as input to generate a gait compact image. Next, gait features are extracted with the optimal feature map by Principal Component Analysis (PCA) and then a linear Support Vector Machine (SVM) is used as the one-against-all multiclass classifier. Four gait compact images including Gait Energy Image (GEI), Gait Entropy Image (GEnI), Gait Gaussian Image (GGI) and the novel gait images called Gait Gaussian Entropy Image (GGEnI) are used as basic gait representations. Then three secondary gait representations are generated from these basic representations. These include Gradient Histogram Gait Image (GHGI) and two novel gait representations called Convolutional Gait Image (CGI) and Convolutional Gradient Histogram Gait Image (CGHGI). All representations are tested with three main tasks. When people walk, each body part does not have the same locomotion information, for example, there is much more motion in the leg than shoulder motion when walking. Moreover, clothing and carrying objects do not have the same level of affect to every part of the body, for example, a handbag does not generally affect leg motion. This study divides the human body into fourteen different body parts based on height. Body parts and gait representations are combined to solve the three main tasks. Three combined parts techniques which use two different parts to solve the problem are created. The fist is Part Scores Fusion (PSF) which uses the summation score of two models based on each part. The highest summation score model is chosen as the result. The second is Part Image Fusion (PIF) which concatenates two parts into a single image with a 1:1 ratio. The highest scoring model which is generated from image fusion is selected as the result. The third is Multi Region Duplication (MRD) which uses the same idea as PIF, however, the second part’s ratio is increased to 1:2, 1:3 and 1:4. These techniques are tested on the gait recognition by identical view. In conclusion, the general framework is effectively for three main tasks. GHGI-GEI which is generated from full silhouette is the most effective representation for gait recognition by identical view and cross view recognition. GHGI-GGI with lower knee region is the most effective representation for view angle classification. The GHGI-GEI CPI combination between full body and limb parts is the most effective combination on OU-LP-Bag. A more detailed description of each aspect is in the following Chapters