Covariate conscious approach for Gait recognition based upon Zernike moment invariants

Gait recognition i.e. identification of an individual from his/her walking pattern is an emerging field. While existing gait recognition techniques perform satisfactorily in normal walking conditions, there performance tend to suffer drastically with variations in clothing and carrying conditions. In this work, we propose a novel covariate cognizant framework to deal with the presence of such covariates. We describe gait motion by forming a single 2D spatio-temporal template from video sequence, called Average Energy Silhouette image (AESI). Zernike moment invariants (ZMIs) are then computed to screen the parts of AESI infected with covariates. Following this, features are extracted from Spatial Distribution of Oriented Gradients (SDOGs) and novel Mean of Directional Pixels (MDPs) methods. The obtained features are fused together to form the final well-endowed feature set. Experimental evaluation of the proposed framework on three publicly available datasets i.e. CASIA dataset B, OU-ISIR Treadmill dataset B and USF Human-ID challenge dataset with recently published gait recognition approaches, prove its superior performance.Comment: 11 page

Human Gait Recognition Subject to Different Covariate Factors in a Multi-View Environment

Human gait recognition system identifies individuals based on their biometric traits. A human’s biometric features can be grouped into physiologic or behavioral traits. Biometric traits, such as the face [1], ears [2], iris [3], finger prints, passwords, and tokens, require highly accurate recognition and a well-controlled human interaction to be effective. In contrast, behavioral traits such as voice, signature, and gait do not require any human interaction and can be collected in a hidden and non-invasive mode with a camera system at a low resolution. In comparison with other physiological traits, one of the main advantages of gait analysis is the collection of data from a certain distance. However, gait is less powerful than physiological traits, yet it still has widespread application in surveillance for unfavorable situations. From traditional algorithms to deep learning models, a gait survey provides a detailed history of gait recognition

Image Enhancement via Deep Spatial and Temporal Networks

Image enhancement is a classic problem in computer vision and has been studied for decades. It includes various subtasks such as super-resolution, image deblurring, rain removal and denoise. Among these tasks, image deblurring and rain removal have become increasingly active, as they play an important role in many areas such as autonomous driving, video surveillance and mobile applications. In addition, there exists connection between them. For example, blur and rain often degrade images simultaneously, and the performance of their removal rely on the spatial and temporal learning. To help generate sharp images and videos, in this thesis, we propose efficient algorithms based on deep neural networks for solving the problems of image deblurring and rain removal. In the first part of this thesis, we study the problem of image deblurring. Four deep learning based image deblurring methods are proposed. First, for single image deblurring, a new framework is presented which firstly learns how to transfer sharp images to realistic blurry images via a learning-to-blur Generative Adversarial Network (GAN) module, and then trains a learning-to-deblur GAN module to learn how to generate sharp images from blurry versions. In contrast to prior work which solely focuses on learning to deblur, the proposed method learns to realistically synthesize blurring effects using unpaired sharp and blurry images. Second, for video deblurring, spatio-temporal learning and adversarial training methods are used to recover sharp and realistic video frames from input blurry versions. 3D convolutional kernels on the basis of deep residual neural networks are employed to capture better spatio-temporal features, and train the proposed network with both the content loss and adversarial loss to drive the model to generate realistic frames. Third, the problem of extracting sharp image sequences from a single motion-blurred image is tackled. A detail-aware network is presented, which is a cascaded generator to handle the problems of ambiguity, subtle motion and loss of details. Finally, this thesis proposes a level-attention deblurring network, and constructs a new large-scale dataset including images with blur caused by various factors. We use this dataset to evaluate current deep deblurring methods and our proposed method. In the second part of this thesis, we study the problem of image deraining. Three deep learning based image deraining methods are proposed. First, for single image deraining, the problem of joint removal of raindrops and rain streaks is tackled. In contrast to most of prior works which solely focus on the raindrops or rain streaks removal, a dual attention-in-attention model is presented, which removes raindrops and rain streaks simultaneously. Second, for video deraining, a novel end-to-end framework is proposed to obtain the spatial representation, and temporal correlations based on ResNet-based and LSTM-based architectures, respectively. The proposed method can generate multiple deraining frames at a time, which outperforms the state-of-the-art methods in terms of quality and speed. Finally, for stereo image deraining, a deep stereo semantic-aware deraining network is proposed for the first time in computer vision. Different from the previous methods which only learn from pixel-level loss function or monocular information, the proposed network advances image deraining by leveraging semantic information and visual deviation between two views

Investigation of gait representations and partial body gait recognition

Recognising an individual by the way they walk is one of the most popular research subjects within the field of soft biometrics in last few decades. The advancement of technology and equipment such as Close Circuit Television (CCTV), wireless internet and wearable sensors makes it easier to obtain gait data than ever before. The gait biometric can be used widely and in different areas such as biomedical, forensic and surveillance. However, gait recognition still has many challenges and fundamental issues. All of these problems only serve as a researcher’s motivation to learn more about various gait topics to overcome the challenges and improve the field of gait recognition. Gait recognition currently has high performance when carried out under very specific conditions such as normal walking, obstruction from certain types of clothing and fixed camera view angles. When the aforementioned conditions are changed, the classification rate dramatically drops. This study aims to solve the problems of clothing, carrying objects and camera view angles within the indoor environment and video-based data collection. Two gait related databases used for testing in this study are CASIA dataset B and OU-ISIR Large population dataset with Bag (OU-LP-Bag). Three main tasks will be tested with CASIA dataset B while only gait recognition is tested with OU-LP-Bag. The gait recognition framework is developed to solve the three main tasks including gait recognition by identical view, view classification and cross view recognition. This framework uses gait images sequence as input to generate a gait compact image. Next, gait features are extracted with the optimal feature map by Principal Component Analysis (PCA) and then a linear Support Vector Machine (SVM) is used as the one-against-all multiclass classifier. Four gait compact images including Gait Energy Image (GEI), Gait Entropy Image (GEnI), Gait Gaussian Image (GGI) and the novel gait images called Gait Gaussian Entropy Image (GGEnI) are used as basic gait representations. Then three secondary gait representations are generated from these basic representations. These include Gradient Histogram Gait Image (GHGI) and two novel gait representations called Convolutional Gait Image (CGI) and Convolutional Gradient Histogram Gait Image (CGHGI). All representations are tested with three main tasks. When people walk, each body part does not have the same locomotion information, for example, there is much more motion in the leg than shoulder motion when walking. Moreover, clothing and carrying objects do not have the same level of affect to every part of the body, for example, a handbag does not generally affect leg motion. This study divides the human body into fourteen different body parts based on height. Body parts and gait representations are combined to solve the three main tasks. Three combined parts techniques which use two different parts to solve the problem are created. The fist is Part Scores Fusion (PSF) which uses the summation score of two models based on each part. The highest summation score model is chosen as the result. The second is Part Image Fusion (PIF) which concatenates two parts into a single image with a 1:1 ratio. The highest scoring model which is generated from image fusion is selected as the result. The third is Multi Region Duplication (MRD) which uses the same idea as PIF, however, the second part’s ratio is increased to 1:2, 1:3 and 1:4. These techniques are tested on the gait recognition by identical view. In conclusion, the general framework is effectively for three main tasks. GHGI-GEI which is generated from full silhouette is the most effective representation for gait recognition by identical view and cross view recognition. GHGI-GGI with lower knee region is the most effective representation for view angle classification. The GHGI-GEI CPI combination between full body and limb parts is the most effective combination on OU-LP-Bag. A more detailed description of each aspect is in the following Chapters