Person recognition based on deep gait: a survey.

Gait recognition, also known as walking pattern recognition, has expressed deep interest in the computer vision and biometrics community due to its potential to identify individuals from a distance. It has attracted increasing attention due to its potential applications and non-invasive nature. Since 2014, deep learning approaches have shown promising results in gait recognition by automatically extracting features. However, recognizing gait accurately is challenging due to the covariate factors, complexity and variability of environments, and human body representations. This paper provides a comprehensive overview of the advancements made in this field along with the challenges and limitations associated with deep learning methods. For that, it initially examines the various gait datasets used in the literature review and analyzes the performance of state-of-the-art techniques. After that, a taxonomy of deep learning methods is presented to characterize and organize the research landscape in this field. Furthermore, the taxonomy highlights the basic limitations of deep learning methods in the context of gait recognition. The paper is concluded by focusing on the present challenges and suggesting several research directions to improve the performance of gait recognition in the future

Human Gait Recognition Under Changes of Walking Conditions

University of Technology Sydney. Faculty of Engineering and Information Technology.Gait has been gathering extensive research interest for its non-fungible position in applications. First, it is difficult to disguise one's gait, since walking is necessary for human mobility. Second, it can be attained at a distance without physical contact or proximal sensing. However, although recently different methods have been proposed for gait recognition, gait analysis is still in its infancy. Most methods enable to garner a remarkable recognition performance when the gallery and the probe are in a similar situation. However, when exterior factors affect a person's gait and changes occur in human appearances, a significant performance degradation happens. Among these exterior factors, clothing variations and mode changes can be treated as the most influential factors for gait recognition. Clothing variations can significantly influence available features to be used in the future recognition process, while walking/running modes can change human motions made by limbs and thus dramatically influence the instinct walking patterns of each person. Thus, in this thesis different methods have been proposed for gait recognition to handle the difficulties of clothing variations and walking/running mode changes. First, given that model-based methods are less vulnerable to clothing variances, a more robust model-based gait feature, Skeleton Gait Energy Image (SGEI), is formed to handle this cloth-changing gait recognition problem. Then, since clothing changes can cause different impacts to different body parts, a part-based collaborative spatio-temporal feature learning method is also proposed for cloth-changing gait recognition by concatenating features from the non/less affected body parts under the correlative H-W and T-W views. Based on the aforementioned two methods, another efficient network is proposed for cloth-changing gait recognition. This network consists of two sub-networks, aiming to produce part-based features from the non/less affected body parts and the estimated skeleton key-point regions. Moreover, in order to address the walking-vs-running problem in a cross-mode manner, a feasible hybrid method is also proposed in this thesis. Distinct from most cross-mode gait recognition methods, this method focuses on learning mode-invariant features for each person from their innate patterns between walking and running modes. Multi-task learning strategies are also used to enhance the efficiency of these learned features. Finally, given that the above-mentioned methods are all proposed based on sufficient input data, a complementary solution is given when only a few gait frames can be offered. Experiments have proved that these proposed methods can obtain a remarkable performance when tackling the cloth-changing and walking-vs-running gait recognition problems

DyGait: Exploiting Dynamic Representations for High-performance Gait Recognition

Gait recognition is a biometric technology that recognizes the identity of humans through their walking patterns. Compared with other biometric technologies, gait recognition is more difficult to disguise and can be applied to the condition of long-distance without the cooperation of subjects. Thus, it has unique potential and wide application for crime prevention and social security. At present, most gait recognition methods directly extract features from the video frames to establish representations. However, these architectures learn representations from different features equally but do not pay enough attention to dynamic features, which refers to a representation of dynamic parts of silhouettes over time (e.g. legs). Since dynamic parts of the human body are more informative than other parts (e.g. bags) during walking, in this paper, we propose a novel and high-performance framework named DyGait. This is the first framework on gait recognition that is designed to focus on the extraction of dynamic features. Specifically, to take full advantage of the dynamic information, we propose a Dynamic Augmentation Module (DAM), which can automatically establish spatial-temporal feature representations of the dynamic parts of the human body. The experimental results show that our DyGait network outperforms other state-of-the-art gait recognition methods. It achieves an average Rank-1 accuracy of 71.4% on the GREW dataset, 66.3% on the Gait3D dataset, 98.4% on the CASIA-B dataset and 98.3% on the OU-MVLP dataset

Deep Learning Based Abnormal Gait Classification System Study with Heterogeneous Sensor Network

Gait is one of the important biological characteristics of the human body. Abnormal gait is mostly related to the lesion site and has been demonstrated to play a guiding role in clinical research such as medical diagnosis and disease prevention. In order to promote the research of automatic gait pattern recognition, this paper introduces the research status of abnormal gait recognition and systems analysis of the common gait recognition technologies. Based on this, two gait information extraction methods, sensor-based and vision-based, are studied, including wearable system design and deep neural network-based algorithm design. In the sensor-based study, we proposed a lower limb data acquisition system. The experiment was designed to collect acceleration signals and sEMG signals under normal and pathological gaits. Specifically, wearable hardware-based on MSP430 and upper computer software based on Labview is designed. The hardware system consists of EMG foot ring, high-precision IMU and pressure-sensitive intelligent insole. Data of 15 healthy persons and 15 hemiplegic patients during walking were collected. The classification of gait was carried out based on sEMG and the average accuracy rate can reach 92.8% for CNN. For IMU signals five kinds of abnormal gait are trained based on three models: BPNN, LSTM, and CNN. The experimental results show that the system combined with the neural network can classify different pathological gaits well, and the average accuracy rate of the six-classifications task can reach 93%. In vision-based research, by using human keypoint detection technology, we obtain the precise location of the key points through the fusion of thermal mapping and offset, thus extracts the space-time information of the key points. However, the results show that even the state-of-the-art is not good enough for replacing IMU in gait analysis and classification. The good news is the rhythm wave can be observed within 2 m, which proves that the temporal and spatial information of the key points extracted is highly correlated with the acceleration information collected by IMU, which paved the way for the visual-based abnormal gait classification algorithm.步态指人走路时表现出来的姿态，是人体重要生物特征之一。异常步态多与病变部位有关，作为反映人体健康状况和行为能力的重要特征，其被论证在医疗诊断、疾病预防等临床研究中具有指导作用。为了促进步态模式自动识别的研究，本文介绍了异常步态识别的研究现状，系统地分析了常见步态识别技术以及算法，以此为基础研究了基于传感器与基于视觉两种步态信息提取方法，内容包括可穿戴系统设计与基于深度神经网络的算法设计。 在基于传感器的研究中，本工作开发了下肢步态信息采集系统，并利用该信息采集系统设计实验，采集正常与不同病理步态下的加速度信号与肌电信号，搭建深度神经网络完成分类任务。具体的，在系统搭建部分设计了基于MSP430的可穿戴硬件设备以及基于Labview的上位机软件，该硬件系统由肌电脚环，高精度IMU以及压感智能鞋垫组成，该上位机软件接收、解包蓝牙数据并计算出步频步长等常用步态参数。 在基于运动信号与基于表面肌电的研究中，采集了15名健康人与15名偏瘫病人的步态数据，并针对表面肌电信号训练卷积神经网络进行帕金森步态的识别与分类，平均准确率可达92.8%。针对运动信号训练了反向传播神经网络，LSTM以及卷积神经网络三种模型进行五种异常步态的分类任务。实验结果表明，本工作中步态信息采集系统结合神经网络模型，可以很好地对不同病理步态进行分类，六分类平均正确率可达93%。 在基于视觉的研究中，本文利用人体关键点检测技术，首先检测出图片中的一个或多个人，接着对边界框做图像分割，接着采用全卷积resnet对每一个边界框中的人物的主要关节点做热力图并分析偏移量，最后通过热力图与偏移的融合得到关键点的精确定位。通过该算法提取了不同步态下姿态关键点时空信息，为基于视觉的步态分析系统提供了基础条件。但实验结果表明目前最高准确率的人体关键点检测算法不足以替代IMU实现步态分析与分类。但在2m之内可以观察到节律信息，证明了所提取的关键点时空信息与IMU采集的加速度信息呈现较高相关度，为基于视觉的异常步态分类算法铺平了道路

Physical Adversarial Attacks for Surveillance: A Survey

Modern automated surveillance techniques are heavily reliant on deep learning methods. Despite the superior performance, these learning systems are inherently vulnerable to adversarial attacks - maliciously crafted inputs that are designed to mislead, or trick, models into making incorrect predictions. An adversary can physically change their appearance by wearing adversarial t-shirts, glasses, or hats or by specific behavior, to potentially avoid various forms of detection, tracking and recognition of surveillance systems; and obtain unauthorized access to secure properties and assets. This poses a severe threat to the security and safety of modern surveillance systems. This paper reviews recent attempts and findings in learning and designing physical adversarial attacks for surveillance applications. In particular, we propose a framework to analyze physical adversarial attacks and provide a comprehensive survey of physical adversarial attacks on four key surveillance tasks: detection, identification, tracking, and action recognition under this framework. Furthermore, we review and analyze strategies to defend against the physical adversarial attacks and the methods for evaluating the strengths of the defense. The insights in this paper present an important step in building resilience within surveillance systems to physical adversarial attacks

A Review of Techniques on Gait-Based Person Re-Identification

Copyright (c) 2023 Babak Rahi, Maozhen Li and Man Qi. Person re-identification at a distance across multiple non-overlapping cameras has been an active research area for years. In the past ten years, short-term Person re-identification techniques have made great strides in accuracy using only appearance features in limited environments. However, massive intra-class variations and inter-class confusion limit their ability to be used in practical applications. Moreover, appearance consistency can only be assumed in a short time span from one camera to the other. Since the holistic appearance will change drastically over days and weeks, the technique, as mentioned above, will be ineffective. Practical applications usually require a long-term solution in which the subject's appearance and clothing might have changed after the elapse of a significant period. Facing these problems, soft biometric features such as Gait has stirred much interest in the past years. Nevertheless, even Gait can vary with illness, ageing and emotional states, walking surfaces, shoe types, clothes types, carried objects (by the subject) and even environment clutters. Therefore, Gait is considered as a temporal cue that could provide biometric motion information. On the other hand, the shape of the human body could be viewed as a spatial signal which can produce valuable information. So extracting discriminative features from both spatial and temporal domains would benefit this research. This article examines the main approaches used in gait analysis for re-identification over the past decade. We identify several relevant dimensions of the problem and provide a taxonomic analysis of current research. We conclude by reviewing the performance levels achievable with current technology and providing a perspective on the most challenging and promising research directions.This research received no external funding

Learning Rich Features for Gait Recognition by Integrating Skeletons and Silhouettes

Gait recognition captures gait patterns from the walking sequence of an individual for identification. Most existing gait recognition methods learn features from silhouettes or skeletons for the robustness to clothing, carrying, and other exterior factors. The combination of the two data modalities, however, is not fully exploited. Previous multimodal gait recognition methods mainly employ the skeleton to assist the local feature extraction where the intrinsic discrimination of the skeleton data is ignored. This paper proposes a simple yet effective Bimodal Fusion (BiFusion) network which mines discriminative gait patterns in skeletons and integrates with silhouette representations to learn rich features for identification. Particularly, the inherent hierarchical semantics of body joints in a skeleton is leveraged to design a novel Multi-Scale Gait Graph (MSGG) network for the feature extraction of skeletons. Extensive experiments on CASIA-B and OUMVLP demonstrate both the superiority of the proposed MSGG network in modeling skeletons and the effectiveness of the bimodal fusion for gait recognition. Under the most challenging condition of walking in different clothes on CASIA-B, our method achieves the rank-1 accuracy of 92.1%.Comment: The paper is under consideration at Multimedia Tools and Application

Human Body Pose Estimation for Gait Identification: A Comprehensive Survey of Datasets and Models

Person identification is a problem that has received substantial attention, particularly in security domains. Gait recognition is one of the most convenient approaches enabling person identification at a distance without the need of high-quality images. There are several review studies addressing person identification such as the utilization of facial images, silhouette images, and wearable sensor. Despite skeletonbased person identification gaining popularity while overcoming the challenges of traditional approaches, existing survey studies lack the comprehensive review of skeleton-based approaches to gait identification. We present a detailed review of the human pose estimation and gait analysis that make the skeleton-based approaches possible. The study covers various types of related datasets, tools, methodologies, and evaluation metrics with associated challenges, limitations, and application domains. Detailed comparisons are presented for each of these aspects with recommendations for potential research and alternatives. A common trend throughout this paper is the positive impact that deep learning techniques are beginning to have on topics such as human pose estimation and gait identification. The survey outcomes might be useful for the related research community and other stakeholders in terms of performance analysis of existing methodologies, potential research gaps, application domains, and possible contributions in the future