Rapid Extraction of Respiratory Waveforms from Photoplethysmography: A Deep Encoder Approach

Much of the information of breathing is contained within the photoplethysmography (PPG) signal, through changes in venous blood flow, heart rate and stroke volume. We aim to leverage this fact, by employing a novel deep learning framework which is a based on a repurposed convolutional autoencoder. Our model aims to encode all of the relevant respiratory information contained within photoplethysmography waveform, and decode it into a waveform that is similar to a gold standard respiratory reference. The model is employed on two photoplethysmography data sets, namely Capnobase and BIDMC. We show that the model is capable of producing respiratory waveforms that approach the gold standard, while in turn producing state of the art respiratory rate estimates. We also show that when it comes to capturing more advanced respiratory waveform characteristics such as duty cycle, our model is for the most part unsuccessful. A suggested reason for this, in light of a previous study on in-ear PPG, is that the respiratory variations in finger-PPG are far weaker compared with other recording locations. Importantly, our model can perform these waveform estimates in a fraction of a millisecond, giving it the capacity to produce over 6 hours of respiratory waveforms in a single second. Moreover, we attempt to interpret the behaviour of the kernel weights within the model, showing that in part our model intuitively selects different breathing frequencies. The model proposed in this work could help to improve the usefulness of consumer PPG-based wearables for medical applications, where detailed respiratory information is required

Towards Patient-Specific Brain Networks Using Functional Magnetic Resonance Imaging

fMRI applications are rare in translational medicine and clinical practice. What can be inferred from a single fMRI scan is often unreliable due to the relative low signal-to-noise ratio compared to other neuroimaging modalities. However, the potential of fMRI is promising. It is one of the few neuroimaging modalities to obtain functional brain organisation of an individual during task engagement and rest. This work extends on current fMRI image processing approaches to obtain robust estimates of functional brain organisation in two resting-state fMRI cohorts. The first cohort comprises of young adults who were born at extremely low gestations and age-matched healthy controls. Group analysis between term- and preterm-born adults revealed differences in functional organisation, which were discovered to be predominantly caused by underlying structural and physiological differences. The second cohort comprises of elderly adults with young onset Alzheimer’s disease and age-matched controls. Their corresponding resting-state fMRI scans are short in scanning time resulting in unreliable spatial estimates with conventional dual regression analysis. This problem was addressed by the development of an ensemble averaging of matrix factorisations approach to compute single subject spatial maps characterised by improved spatial reproducibility compared to maps obtained by dual regression. The approach was extended with a haemodynamic forward model to obtain surrogate neural activations to examine the subject’s task behaviour. This approach applied to two task-fMRI cohorts showed that these surrogate neural activations matched with original task timings in most of the examined fMRI scans but also revealed subjects with task behaviour different than intended by the researcher. It is hoped that both the findings in this work and the novel matrix factorisation approach itself will benefit the fMRI community. To this end, the derived tools are made available online to aid development and validation of methods for resting-state and task fMRI experiments

Signal processing techniques for cardiovascular monitoring applications using conventional and video-based photoplethysmography

Photoplethysmography (PPG)-based monitoring devices will probably play a decisive role in healthcare environment of the future, which will be preventive, predictive, personalized and participatory. Indeed, this optical technology presents several practical advantages over gold standard methods based on electrocardiography, because PPG wearable devices can be comfortably used for long-term continuous monitoring during daily life activities. Contactless video-based PPG technique, also known as imaging photoplethysmography (iPPG), has also attracted much attention recently. In that case, the cardiac pulse is remotely measured from the subtle skin color changes resulting from the blood circulation, using a simple video camera. PPG/iPPG have a lot of potential for a wide range of cardiovascular applications. Hence, there is a substantial need for signal processing techniques to explore these applications and to improve the reliability of the PPG/iPPG-based parameters. \par A part of the thesis is dedicated to the development of robust processing schemes to estimate heart rate from the PPG/iPPG signals. The proposed approaches were built on adaptive frequency tracking algorithms that were previously developed in our group. These tools, based on adaptive band-pass filters, provide instantaneous frequency estimates of the input signal(s) with a very low time delay, making them suitable for real-time applications. In case of conventional PPG, a prior adaptive noise cancellation step involving the use of accelerometer signals was also necessary to reconstruct clean PPG signals during the regions corrupted by motion artifacts. Regarding iPPG, after comparing different regions of interest on the subject face, we hypothesized that the simultaneous use of different iPPG signal derivation methods (i.e. methods to derive the iPPG time series from the pixel values of the consecutive frames) could be advantageous. Methods to assess signal quality online and to incorporate it into instantaneous frequency estimation were also examined and successfully applied to improve system reliability. \par This thesis also explored different innovative applications involving PPG/iPPG signals. The detection of atrial fibrillation was studied. Novel features derived directly from the PPG waveforms, designed to reflect the morphological changes observed during arrhythmic episodes, were proposed and proven to be successful for atrial fibrillation detection. Arrhythmia detection and robust heart rate estimation approaches were combined in another study aimed at reducing the number of false arrhythmia alarms in the intensive care unit by exploiting signals from independent sources, including PPG. Evaluation on a hidden dataset demonstrated that the number of false alarms was drastically reduced while almost no true alarm was suppressed. Finally, other aspects of the iPPG technology were examined, such as the measurement of pulse rate variability indexes from the iPPG signals and the estimation of respiratory rate from the iPPG interbeat intervals

Sensing and Signal Processing in Smart Healthcare

In the last decade, we have witnessed the rapid development of electronic technologies that are transforming our daily lives. Such technologies are often integrated with various sensors that facilitate the collection of human motion and physiological data and are equipped with wireless communication modules such as Bluetooth, radio frequency identification, and near-field communication. In smart healthcare applications, designing ergonomic and intuitive human–computer interfaces is crucial because a system that is not easy to use will create a huge obstacle to adoption and may significantly reduce the efficacy of the solution. Signal and data processing is another important consideration in smart healthcare applications because it must ensure high accuracy with a high level of confidence in order for the applications to be useful for clinicians in making diagnosis and treatment decisions. This Special Issue is a collection of 10 articles selected from a total of 26 contributions. These contributions span the areas of signal processing and smart healthcare systems mostly contributed by authors from Europe, including Italy, Spain, France, Portugal, Romania, Sweden, and Netherlands. Authors from China, Korea, Taiwan, Indonesia, and Ecuador are also included

Emotion and Stress Recognition Related Sensors and Machine Learning Technologies

This book includes impactful chapters which present scientific concepts, frameworks, architectures and ideas on sensing technologies and machine learning techniques. These are relevant in tackling the following challenges: (i) the field readiness and use of intrusive sensor systems and devices for capturing biosignals, including EEG sensor systems, ECG sensor systems and electrodermal activity sensor systems; (ii) the quality assessment and management of sensor data; (iii) data preprocessing, noise filtering and calibration concepts for biosignals; (iv) the field readiness and use of nonintrusive sensor technologies, including visual sensors, acoustic sensors, vibration sensors and piezoelectric sensors; (v) emotion recognition using mobile phones and smartwatches; (vi) body area sensor networks for emotion and stress studies; (vii) the use of experimental datasets in emotion recognition, including dataset generation principles and concepts, quality insurance and emotion elicitation material and concepts; (viii) machine learning techniques for robust emotion recognition, including graphical models, neural network methods, deep learning methods, statistical learning and multivariate empirical mode decomposition; (ix) subject-independent emotion and stress recognition concepts and systems, including facial expression-based systems, speech-based systems, EEG-based systems, ECG-based systems, electrodermal activity-based systems, multimodal recognition systems and sensor fusion concepts and (x) emotion and stress estimation and forecasting from a nonlinear dynamical system perspective

Simulation and implementation of novel deep learning hardware architectures for resource constrained devices

Corey Lammie designed mixed signal memristive-complementary metal–oxide–semiconductor (CMOS) and field programmable gate arrays (FPGA) hardware architectures, which were used to reduce the power and resource requirements of Deep Learning (DL) systems; both during inference and training. Disruptive design methodologies, such as those explored in this thesis, can be used to facilitate the design of next-generation DL systems

A PROBABILISTIC APPROACH TO THE CONSTRUCTION OF A MULTIMODAL AFFECT SPACE

Understanding affective signals from others is crucial for both human-human and human-agent interaction. The automatic analysis of emotion is by and large addressed as a pattern recognition problem which grounds in early psychological theories of emotion. Suitable features are first extracted and then used as input to classification (discrete emotion recognition) or regression (continuous affect detection). In this thesis, differently from many computational models in the literature, we draw on a simulationist approach to the analysis of facially displayed emotions - e.g., in the course of a face-to-face interaction between an expresser and an observer. At the heart of such perspective lies the enactment of the perceived emotion in the observer. We propose a probabilistic framework based on a deep latent representation of a continuous affect space, which can be exploited for both the estimation and the enactment of affective states in a multimodal space. Namely, we consider the observed facial expression together with physiological activations driven by internal autonomic activity. The rationale behind the approach lies in the large body of evidence from affective neuroscience showing that when we observe emotional facial expressions, we react with congruent facial mimicry. Further, in more complex situations, affect understanding is likely to rely on a comprehensive representation grounding the reconstruction of the state of the body associated with the displayed emotion. We show that our approach can address such problems in a unified and principled perspective, thus avoiding ad hoc heuristics while minimising learning efforts. Moreover, our model improves the inferred belief through the adoption of an inner loop of measurements and predictions within the central affect state-space, that realise the dynamics of the affect enactment. Results so far achieved have been obtained by adopting two publicly available multimodal corpora