Sensors Fusion for Cognitive Load Analysis using Gait Data

ait is the manner of walking in people and one of the basic functions for humans to move purposefully to reach a desired destination. The quality of life can be affected by gait abnormality and result in morbidity and mortality. Substantially, our aim in this research is to develop new methods and algorithms that make the most of the existing sensors for gait analysis. A detailed review [1] reveals the existing achievements and gaps in the current knowledge in gait analysis. The modalities in literature to capture gait data are grouped according to the sensing technology: video sequences, wearable sensors, and floor sensors. Following from the review, sensors under the foot are identified as a suitable method to study gait deterioration due to cognitive load in this research. Therefore, Deep learning models are implemented to fuse sensors under the foot and deliver automatic feature extraction of gait patterns and perform classification for the following. (a) Gait under cognitive load difference in males and females [2], where both genders identified by 95% yet they share the same cognitive load by 93%. (b) Healthy subjects’ natural limits due to cognitive load capacity investigated using their gait. Layer-Wise Relevance Propagation technique is used to link key known events in the gait cycle to identify the influence of cognitive demanding tasks on gait [3]. (c) Parkinson’s disease staging based on postural imbalance caused by gait deterioration. The models classified patients’ gait by 96% using ground truth markers [4]. These findings present valuable insight for gait spatiotemporal signals analysis, with other potential spin-offs are in the areas of biometrics and security

Evaluation of supervised classification algorithms for human activity recognition with inertial sensors

The main aim of this work is to compare the performance of different algorithms for human activity recognition by extracting various statistical time domain and frequency domain features from the inertial sensor data. Our results show that Support Vector Machines with quadratic kernel classifier (accuracy: 93.5%) and Ensemble classifier with bagging and boosting (accuracy: 94.6%) outperforms other known activity classification algorithms. A parallel coordinate plot based on visualization of features is used to identify useful features or predictors for separating classes. This enabled exclusion of features that contribute least to classification accuracy in a multi-sensor system (five in our case), made the classifier lightweight in terms of number of useful features, training time and computational load and lends itself to real-time implementation

Analysis of spatio-temporal representations for robust footstep recognition with deep residual neural networks

IEEE: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.”Human footsteps can provide a unique behavioural pattern for robust biometric systems. We propose spatio-temporal footstep representations from floor-only sensor data in advanced computational models for automatic biometric verification. Our models deliver an artificial intelligence capable of effectively differentiating the fine-grained variability of footsteps between legitimate users (clients) and impostor users of the biometric system. The methodology is validated in the largest to date footstep database, containing nearly 20,000 footstep signals from more than 120 users. The database is organized by considering a large cohort of impostors and a small set of clients to verify the reliability of biometric systems. We provide experimental results in 3 critical data-driven security scenarios, according to the amount of footstep data made available for model training: at airports security checkpoints (smallest training set), workspace environments (medium training set) and home environments (largest training set). We report state-of-the-art footstep recognition rates with an optimal equal false acceptance and false rejection rate of 0.7% (equal error rate), an improvement ratio of 371% from previous state-of-the-art. We perform a feature analysis of deep residual neural networks showing effective clustering of client's footstep data and provide insights of the feature learning process.This work has been partially supported by Cognimetrics TEC2015-70627-R MINECO/FEDE

Spatial footstep recognition by convolutional neural networks for biometrie applications

We propose a Convolutional Neural Network model to learn spatial footstep features end-to-end from a floor sensor system for biometric applications. Our model’s generalization performance is assessed by independent validation and evaluation datasets from the largest footstep database to date, containing nearly 20,000 footstep signals from 127 users. We report footstep recognition performance as Equal Error Rate in the range of 9% to 13% depending on the test set. This improves previously reported footstep recognition rates in the spatial domain up to 4% EE

Design and implementation of a convolutional neural network on an edge computing smartphone for human activity recognition

Edge computing aims to integrate computing into everyday settings, enabling the system to be context-aware and private to the user. With the increasing success and popularity of deep learning methods, there is an increased demand to leverage these techniques in mobile and wearable computing scenarios. In this paper, we present an assessment of a deep human activity recognition system’s memory and execution time requirements, when implemented on a mid-range smartphone class hardware and the memory implications for embedded hardware. This paper presents the design of a convolutional neural network (CNN) in the context of human activity recognition scenario. Here, layers of CNN automate the feature learning and the influence of various hyper-parameters such as the number of filters and filter size on the performance of CNN. The proposed CNN showed increased robustness with better capability of detecting activities with temporal dependence compared to models using statistical machine learning techniques. The model obtained an accuracy of 96.4% in a five-class static and dynamic activity recognition scenario. We calculated the proposed model memory consumption and execution time requirements needed for using it on a mid-range smartphone. Per-channel quantization of weights and per-layer quantization of activation to 8-bits of precision post-training produces classification accuracy within 2% of floating-point networks for dense, convolutional neural network architecture. Almost all the size and execution time reduction in the optimized model was achieved due to weight quantization. We achieved more than four times reduction in model size when optimized to 8-bit, which ensured a feasible model capable of fast on-device inference