An Interpretable Machine Vision Approach to Human Activity Recognition using Photoplethysmograph Sensor Data

The current gold standard for human activity recognition (HAR) is based on the use of cameras. However, the poor scalability of camera systems renders them impractical in pursuit of the goal of wider adoption of HAR in mobile computing contexts. Consequently, researchers instead rely on wearable sensors and in particular inertial sensors. A particularly prevalent wearable is the smart watch which due to its integrated inertial and optical sensing capabilities holds great potential for realising better HAR in a non-obtrusive way. This paper seeks to simplify the wearable approach to HAR through determining if the wrist-mounted optical sensor alone typically found in a smartwatch or similar device can be used as a useful source of data for activity recognition. The approach has the potential to eliminate the need for the inertial sensing element which would in turn reduce the cost of and complexity of smartwatches and fitness trackers. This could potentially commoditise the hardware requirements for HAR while retaining the functionality of both heart rate monitoring and activity capture all from a single optical sensor. Our approach relies on the adoption of machine vision for activity recognition based on suitably scaled plots of the optical signals. We take this approach so as to produce classifications that are easily explainable and interpretable by non-technical users. More specifically, images of photoplethysmography signal time series are used to retrain the penultimate layer of a convolutional neural network which has initially been trained on the ImageNet database. We then use the 2048 dimensional features from the penultimate layer as input to a support vector machine. Results from the experiment yielded an average classification accuracy of 92.3%. This result outperforms that of an optical and inertial sensor combined (78%) and illustrates the capability of HAR systems using...Comment: 26th AIAI Irish Conference on Artificial Intelligence and Cognitive Scienc

Embedding Temporal Convolutional Networks for Energy-efficient PPG-based Heart Rate Monitoring

Photoplethysmography (PPG) sensors allow for non-invasive and comfortable heart rate (HR) monitoring, suitable for compact wrist-worn devices. Unfortunately, motion artifacts (MAs) severely impact the monitoring accuracy, causing high variability in the skin-to-sensor interface. Several data fusion techniques have been introduced to cope with this problem, based on combining PPG signals with inertial sensor data. Until now, both commercial and reasearch solutions are computationally efficient but not very robust, or strongly dependent on hand-tuned parameters, which leads to poor generalization performance. In this work, we tackle these limitations by proposing a computationally lightweight yet robust deep learning-based approach for PPG-based HR estimation. Specifically, we derive a diverse set of Temporal Convolutional Networks for HR estimation, leveraging Neural Architecture Search. Moreover, we also introduce ActPPG, an adaptive algorithm that selects among multiple HR estimators depending on the amount of MAs, to improve energy efficiency. We validate our approaches on two benchmark datasets, achieving as low as 3.84 beats per minute of Mean Absolute Error on PPG-Dalia, which outperforms the previous state of the art. Moreover, we deploy our models on a low-power commercial microcontroller (STM32L4), obtaining a rich set of Pareto optimal solutions in the complexity vs. accuracy space

Peak Detection and HRV Feature Evaluation on ECG and PPG Signals

Publisher Copyright: © 2022 by the authors. Licensee MDPI, Basel, Switzerland.Heart Rate Variability (HRV) evaluates the autonomic nervous system regulation and can be used as a monitoring tool in conditions such as cardiovascular diseases, neuropathies and sleep staging. It can be extracted from the electrocardiogram (ECG) and the photoplethysmogram (PPG) signals. Typically, the HRV is obtained from the ECG processing. Being the PPG sensor widely used in clinical setups for physiological parameters monitoring such as blood oxygenation and ventilatory rate, the question arises regarding the PPG adequacy for HRV extraction. There is not a consensus regarding the PPG being able to replace the ECG in the HRV estimation. This work aims to be a contribution to this research area by comparing the HRV estimation obtained from simultaneously acquired ECG and PPG signals from forty subjects. A peak detection method is herein introduced based on the Hilbert transform: Hilbert Double Envelope Method (HDEM). Two other peak detector methods were also evaluated: Pan-Tompkins and Wavelet-based. HRV parameters for time, frequency and the non-linear domain were calculated for each algorithm and the Pearson correlation, T-test and RMSE were evaluated. The HDEM algorithm showed the best overall results with a sensitivity of 99.07% and 99.45% for the ECG and the PPG signals, respectively. For this algorithm, a high correlation and no significant differences were found between HRV features and the gold standard, for the ECG and PPG signals. The results show that the PPG is a suitable alternative to the ECG for HRV feature extraction.publishersversionpublishe

Q-PPG: Energy-Efficient PPG-Based Heart Rate Monitoring on Wearable Devices

Hearth Rate (HR) monitoring is increasingly performed in wrist-worn devices using low-cost photoplethysmography (PPG) sensors. However, Motion Artifacts (MAs) caused by movements of the subject's arm affect the performance of PPG-based HR tracking. This is typically addressed coupling the PPG signal with acceleration measurements from an inertial sensor. Unfortunately, most standard approaches of this kind rely on hand-tuned parameters, which impair their generalization capabilities and their applicability to real data in the field. In contrast, methods based on deep learning, despite their better generalization, are considered to be too complex to deploy on wearable devices.In this work, we tackle these limitations, proposing a design space exploration methodology to automatically generate a rich family of deep Temporal Convolutional Networks (TCNs) for HR monitoring, all derived from a single "seed" model. Our flow involves a cascade of two Neural Architecture Search (NAS) tools and a hardware-friendly quantizer, whose combination yields both highly accurate and extremely lightweight models. When tested on the PPG-Dalia dataset, our most accurate model sets a new state-of-the-art in Mean Absolute Error. Furthermore, we deploy our TCNs on an embedded platform featuring a STM32WB55 microcontroller, demonstrating their suitability for real-time execution. Our most accurate quantized network achieves 4.41 Beats Per Minute (BPM) of Mean Absolute Error (MAE), with an energy consumption of 47.65 mJ and a memory footprint of 412 kB. At the same time, the smallest network that obtains a MAE < 8 BPM, among those generated by our flow, has a memory footprint of 1.9 kB and consumes just 1.79 mJ per inference

Transcending conventional biometry frontiers: Diffusive Dynamics PPG Biometry

In the first half of the 20th century, a first pulse oximeter was available to measure blood flow changes in the peripheral vascular net. However, it was not until recent times the PhotoPlethysmoGraphic (PPG) signal used to monitor many physiological parameters in clinical environments. Over the last decade, its use has extended to the area of biometrics, with different methods that allow the extraction of characteristic features of each individual from the PPG signal morphology, highly varying with time and the physical states of the subject. In this paper, we present a novel PPG-based biometric authentication system based on convolutional neural networks. Contrary to previous approaches, our method extracts the PPG signal's biometric characteristics from its diffusive dynamics, characterized by geometric patterns image in the (p, q)-planes specific to the 0-1 test. The diffusive dynamics of the PPG signal are strongly dependent on the vascular bed's biostructure, which is unique to each individual, and highly stable over time and other psychosomatic conditions. Besides its robustness, our biometric method is anti-spoofing, given the convoluted nature of the blood network. Our biometric authentication system reaches very low Equal Error Rates (ERRs) with a single attempt, making it possible, by the very nature of the envisaged solution, to implement it in miniature components easily integrated into wearable biometric systems.Comment: 18 pages, 6 figures, 4 table

PPG2ABP: Translating Photoplethysmogram (PPG) Signals to Arterial Blood Pressure (ABP) Waveforms using Fully Convolutional Neural Networks

Cardiovascular diseases are one of the most severe causes of mortality, taking a heavy toll of lives annually throughout the world. The continuous monitoring of blood pressure seems to be the most viable option, but this demands an invasive process, bringing about several layers of complexities. This motivates us to develop a method to predict the continuous arterial blood pressure (ABP) waveform through a non-invasive approach using photoplethysmogram (PPG) signals. In addition we explore the advantage of deep learning as it would free us from sticking to ideally shaped PPG signals only, by making handcrafted feature computation irrelevant, which is a shortcoming of the existing approaches. Thus, we present, PPG2ABP, a deep learning based method, that manages to predict the continuous ABP waveform from the input PPG signal, with a mean absolute error of 4.604 mmHg, preserving the shape, magnitude and phase in unison. However, the more astounding success of PPG2ABP turns out to be that the computed values of DBP, MAP and SBP from the predicted ABP waveform outperforms the existing works under several metrics, despite that PPG2ABP is not explicitly trained to do so

Advanced Signal Processing in Wearable Sensors for Health Monitoring

Smart, wearables devices on a miniature scale are becoming increasingly widely available, typically in the form of smart watches and other connected devices. Consequently, devices to assist in measurements such as electroencephalography (EEG), electrocardiogram (ECG), electromyography (EMG), blood pressure (BP), photoplethysmography (PPG), heart rhythm, respiration rate, apnoea, and motion detection are becoming more available, and play a significant role in healthcare monitoring. The industry is placing great emphasis on making these devices and technologies available on smart devices such as phones and watches. Such measurements are clinically and scientifically useful for real-time monitoring, long-term care, and diagnosis and therapeutic techniques. However, a pertaining issue is that recorded data are usually noisy, contain many artefacts, and are affected by external factors such as movements and physical conditions. In order to obtain accurate and meaningful indicators, the signal has to be processed and conditioned such that the measurements are accurate and free from noise and disturbances. In this context, many researchers have utilized recent technological advances in wearable sensors and signal processing to develop smart and accurate wearable devices for clinical applications. The processing and analysis of physiological signals is a key issue for these smart wearable devices. Consequently, ongoing work in this field of study includes research on filtration, quality checking, signal transformation and decomposition, feature extraction and, most recently, machine learning-based methods