Multimodal Human Pose Feature Fusion for Gait Recognition.

Gait recognition allows identifying people at a distance based on the way they walk (i.e. gait) in a non-invasive approach. Most of the approaches published in the last decades are dominated by the use of silhouettes or other appearance-based modalities to describe the Gait cycle. In an attempt to exclude the appearance data, many works have been published that address the use of the human pose as a modality to describe the walking movement. However, as the pose contains less information when used as a single modality, the performance achieved by the models is generally poorer. To overcome such limitations, we propose a multimodal setup that combines multiple pose representation models. To this end, we evaluate multiple fusion strategies to aggregate the features derived from each pose modality at every model stage. Moreover, we introduce a weighted sum with trainable weights that can adaptively learn the optimal balance among pose modalities. Our experimental results show that (a) our fusion strategies can effectively combine different pose modalities by improving their baseline performance; and, (b) by using only human pose, our approach outperforms most of the silhouette-based state-of-the-art approaches. Concretely, we obtain 92.8% mean Top-1 accuracy in CASIA-B.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Gait Recognition for Co-Existing Multiple People Using Millimeter Wave Sensing

Gait recognition, i.e., recognizing persons from their walking postures, has found versatile applications in security check, health monitoring, and novel human-computer interaction. The millimeter-wave (mmWave) based gait recognition represents the most recent advance. Compared with traditional camera-based solutions, mmWave based gait recognition bears unique advantages of being still effective under non-line-of-sight scenarios, such as in black, weak light, or blockage conditions. Moreover, they are able to accomplish person identification while preserving privacy. Currently, there are only few works in mmWave gait recognition, since no public data set is available. In this paper, we build a first-of-its-kind mmWave gait data set, in which we collect gait of 95 volunteers 'seen' from two mmWave radars in two different scenarios, which together lasts about 30 hours. Using the data set, we propose a novel deep-learning driven mmWave gait recognition method called mmGaitNet, and compare it with five state-of-the-art algorithms. We find that mmGaitNet is able to achieve 90% accuracy for single-person scenarios, 88% accuracy for five co-existing persons, while the existing methods achieve less than 66% accuracy for both scenarios

Radar sensing for ambient assisted living application with artificial intelligence

In a time characterized by rapid technological advancements and a noticeable trend towards an older average population, the need for automated systems to monitor movements and actions has become increasingly important. This thesis delves into the application of radar, specifically Frequency Modulated Continuous Wave (FMCW) radar, as an emerging and effective sensor in the field of "Activity Recognition." This area involves capturing motion data through sensors and integrating it with machine learning algorithms to autonomously classify human activities. Radar is distinguished by its ability to accurately track complex bodily movements while ensuring privacy compliance. The research provides an in-depth examination of FMCW radar, detailing its operational principles and exploring radar information domains such as range-time and micro-Doppler signatures. Following this, the thesis presents a state-of-the-art review in activity recognition, discussing key papers and significant works that have shaped the field. The thesis then focuses on research topics where contributions were made. The first topic is human activity recognition (HAR) with different physiology, presenting a comprehensive experimental setup with radar sensors to capture various human activities. The analysis of classification results reveals the effectiveness of different radar representations. Advancing into the domain of resource-constrained system platforms. It introduces adaptive thresholding for efficient data processing and discusses the optimization of these methods using artificial intelligence, particularly focusing on the evolution algorithm such as Self-Adaptive Differential Evolution Algorithm (SADEA). The final chapter discusses the use of Long Short-Term Memory (LSTM) networks for short-range personnel recognition using radar signals. It details the training and testing methodologies and provides an analysis of LSTM networks performance in temporal classification tasks. Overall, this thesis demonstrates the effectiveness of merging radar technology with machine learning in HAR, particularly in assisted living. It contributes to the field by introducing methods optimized for resource-limited settings and innovative approaches in temporal classification using LSTM networks