Extraction of Heart Rate from Multimodal Video Streams of Neonates using Methods of Machine Learning

The World Health Organization estimates that more than one-tenth of births are premature. Premature births are linked to an increase of the mortality risk, when compared with full-term infants. In fact, preterm birth complications are the leading cause of perinatal mortality. These complications range from respiratory distress to cardiovascular disorders. Vital signs changes are often prior to these major complications, therefore it is crucial to perform continuous monitoring of this signals. Heart rate monitoring is particularly important. Nowadays, the standard method to monitor this vital sign requires adhesive electrodes or sensors that are attached to the infant. This contact-based methods can damage the skin of the infant, possibly leading to infections. Within this context, there is a need to evolve to remote heart rate monitoring methods. This thesis introduces a new method for region of interest selection to improve remote heart rate monitoring in neonatology through Photoplethysmography Imaging. The heart rate assessment is based on the standard photoplethysmography principle, which makes use of the subtle fluctuations of visible or infrared light that is reflected from the skin surface within the cardiac cycle. A camera is used, instead of the contact-based sensors. Specifically, this thesis presents an alternative method to manual region of interest selection using methods of Machine Learning, aiming to improve the robustness of Photoplethysmography Imaging. This method comprises a highly efficient Fully Convolutional Neural Network to select six different body regions, within each video frame. The developed neural network was built upon a ResNet network and a custom upsampling network. Additionally, a new post-processing method was developed to refine the body segmentation results, using a sequence of morphological operations and centre of mass analysis. The developed region of interest selection method was validated with clinical data, demonstrating a good agreement (78%) between the estimated heart rate and the reference

Respiration rate and volume measurements using wearable strain sensors.

Current methods for continuous respiration monitoring such as respiratory inductive or optoelectronic plethysmography are limited to clinical or research settings; most wearable systems reported only measures respiration rate. Here we introduce a wearable sensor capable of simultaneously measuring both respiration rate and volume with high fidelity. Our disposable respiration sensor with a Band-Aid© like formfactor can measure both respiration rate and volume by simply measuring the local strain of the ribcage and abdomen during breathing. We demonstrate that both metrics are highly correlated to measurements from a medical grade continuous spirometer on participants at rest. Additionally, we also show that the system is capable of detecting respiration under various ambulatory conditions. Because these low-powered piezo-resistive sensors can be integrated with wireless Bluetooth units, they can be useful in monitoring patients with chronic respiratory diseases in everyday settings

Imaging photoplethysmography: towards effective physiological measurements

Since its conception decades ago, Photoplethysmography (PPG) the non-invasive opto-electronic technique that measures arterial pulsations in-vivo has proven its worth by achieving and maintaining its rank as a compulsory standard of patient monitoring. However successful, conventional contact monitoring mode is not suitable in certain clinical and biomedical situations, e.g., in the case of skin damage, or when unconstrained movement is required. With the advance of computer and photonics technologies, there has been a resurgence of interest in PPG and one potential route to overcome the abovementioned issues has been increasingly explored, i.e., imaging photoplethysmography (iPPG). The emerging field of iPPG offers some nascent opportunities in effective and comprehensive interpretation of the physiological phenomena, indicating a promising alternative to conventional PPG. Heart and respiration rate, perfusion mapping, and pulse rate variability have been accessed using iPPG. To effectively and remotely access physiological information through this emerging technique, a number of key issues are still to be addressed. The engineering issues of iPPG, particularly the influence of motion artefacts on signal quality, are addressed in this thesis, where an engineering model based on the revised Beer-Lambert law was developed and used to describe opto-physiological phenomena relevant to iPPG. An iPPG setup consisting of both hardware and software elements was developed to investigate its reliability and reproducibility in the context of effective remote physiological assessment. Specifically, a first study was conducted for the acquisition of vital physiological signs under various exercise conditions, i.e. resting, light and heavy cardiovascular exercise, in ten healthy subjects. The physiological parameters derived from the images captured by the iPPG system exhibited functional characteristics comparable to conventional contact PPG, i.e., maximum heart rate difference was <3 bpm and a significant (p < 0.05) correlation between both measurements were also revealed. Using a method for attenuation of motion artefacts, the heart rate and respiration rate information was successfully assessed from different anatomical locations even in high-intensity physical exercise situations. This study thereby leads to a new avenue for noncontact sensing of vital signs and remote physiological assessment, showing clear and promising applications in clinical triage and sports training. A second study was conducted to remotely assess pulse rate variability (PRV), which has been considered a valuable indicator of autonomic nervous system (ANS) status. The PRV information was obtained using the iPPG setup to appraise the ANS in ten normal subjects. The performance of the iPPG system in accessing PRV was evaluated via comparison with the readings from a contact PPG sensor. Strong correlation and good agreement between these two techniques verify the effectiveness of iPPG in the remote monitoring of PRV, thereby promoting iPPG as a potential alternative to the interpretation of physiological dynamics related to the ANS. The outcomes revealed in the thesis could present the trend of a robust non-contact technique for cardiovascular monitoring and evaluation

The 2023 wearable photoplethysmography roadmap

Photoplethysmography is a key sensing technology which is used in wearable devices such as smartwatches and fitness trackers. Currently, photoplethysmography sensors are used to monitor physiological parameters including heart rate and heart rhythm, and to track activities like sleep and exercise. Yet, wearable photoplethysmography has potential to provide much more information on health and wellbeing, which could inform clinical decision making. This Roadmap outlines directions for research and development to realise the full potential of wearable photoplethysmography. Experts discuss key topics within the areas of sensor design, signal processing, clinical applications, and research directions. Their perspectives provide valuable guidance to researchers developing wearable photoplethysmography technology

Remote Assessment of the Cardiovascular Function Using Camera-Based Photoplethysmography

Camera-based photoplethysmography (cbPPG) is a novel measurement technique that allows the continuous monitoring of vital signs by using common video cameras. In the last decade, the technology has attracted a lot of attention as it is easy to set up, operates remotely, and offers new diagnostic opportunities. Despite the growing interest, cbPPG is not completely established yet and is still primarily the object of research. There are a variety of reasons for this lack of development including that reliable and autonomous hardware setups are missing, that robust processing algorithms are needed, that application fields are still limited, and that it is not completely understood which physiological factors impact the captured signal. In this thesis, these issues will be addressed. A new and innovative measuring system for cbPPG was developed. In the course of three large studies conducted in clinical and non-clinical environments, the system’s great flexibility, autonomy, user-friendliness, and integrability could be successfully proven. Furthermore, it was investigated what value optical polarization filtration adds to cbPPG. The results show that a perpendicular filter setting can significantly enhance the signal quality. In addition, the performed analyses were used to draw conclusions about the origin of cbPPG signals: Blood volume changes are most likely the defining element for the signal's modulation. Besides the hardware-related topics, the software topic was addressed. A new method for the selection of regions of interest (ROIs) in cbPPG videos was developed. Choosing valid ROIs is one of the most important steps in the processing chain of cbPPG software. The new method has the advantage of being fully automated, more independent, and universally applicable. Moreover, it suppresses ballistocardiographic artifacts by utilizing a level-set-based approach. The suitability of the ROI selection method was demonstrated on a large and challenging data set. In the last part of the work, a potentially new application field for cbPPG was explored. It was investigated how cbPPG can be used to assess autonomic reactions of the nervous system at the cutaneous vasculature. The results show that changes in the vasomotor tone, i.e. vasodilation and vasoconstriction, reflect in the pulsation strength of cbPPG signals. These characteristics also shed more light on the origin problem. Similar to the polarization analyses, they support the classic blood volume theory. In conclusion, this thesis tackles relevant issues regarding the application of cbPPG. The proposed solutions pave the way for cbPPG to become an established and widely accepted technology

Characterization and processing of novel neck photoplethysmography signals for cardiorespiratory monitoring

Epilepsy is a neurological disorder causing serious brain seizures that severely affect the patients' quality of life. Sudden unexpected death in epilepsy (SUDEP), for which no evident decease reason is found after post-mortem examination, is a common cause of mortality. The mechanisms leading to SUDEP are uncertain, but, centrally mediated apneic respiratory dysfunction, inducing dangerous hypoxemia, plays a key role. Continuous physiological monitoring appears as the only reliable solution for SUDEP prevention. However, current seizure-detection systems do not show enough sensitivity and present a high number of intolerable false alarms. A wearable system capable of measuring several physiological signals from the same body location, could efficiently overcome these limitations. In this framework, a neck wearable apnea detection device (WADD), sensing airflow through tracheal sounds, was designed. Despite the promising performance, it is still necessary to integrate an oximeter sensor into the system, to measure oxygen saturation in blood (SpO2) from neck photoplethysmography (PPG) signals, and hence, support the apnea detection decision. The neck is a novel PPG measurement site that has not yet been thoroughly explored, due to numerous challenges. This research work aims to characterize neck PPG signals, in order to fully exploit this alternative pulse oximetry location, for precise cardiorespiratory biomarkers monitoring. In this thesis, neck PPG signals were recorded, for the first time in literature, in a series of experiments under different artifacts and respiratory conditions. Morphological and spectral characteristics were analyzed in order to identify potential singularities of the signals. The most common neck PPG artifacts critically corrupting the signal quality, and other breathing states of interest, were thoroughly characterized in terms of the most discriminative features. An algorithm was further developed to differentiate artifacts from clean PPG signals. Both, the proposed characterization and classification model can be useful tools for researchers to denoise neck PPG signals and exploit them in a variety of clinical contexts. In addition to that, it was demonstrated that the neck also offered the possibility, unlike other body parts, to extract the Jugular Venous Pulse (JVP) non-invasively. Overall, the thesis showed how the neck could be an optimum location for multi-modal monitoring in the context of diseases affecting respiration, since it not only allows the sensing of airflow related signals, but also, the breathing frequency component of the PPG appeared more prominent than in the standard finger location. In this context, this property enabled the extraction of relevant features to develop a promising algorithm for apnea detection in near-real time. These findings could be of great importance for SUDEP prevention, facilitating the investigation of the mechanisms and risk factors associated to it, and ultimately reduce epilepsy mortality.Open Acces

Sources of inaccuracy in photoplethysmography for continuous cardiovascular monitoring

Photoplethysmography (PPG) is a low-cost, noninvasive optical technique that uses change in light transmission with changes in blood volume within tissue to provide information for cardiovascular health and fitness. As remote health and wearable medical devices become more prevalent, PPG devices are being developed as part of wearable systems to monitor parameters such as heart rate (HR) that do not require complex analysis of the PPG waveform. However, complex analyses of the PPG waveform yield valuable clinical information, such as: blood pressure, respiratory information, sympathetic nervous system activity, and heart rate variability. Systems aiming to derive such complex parameters do not always account for realistic sources of noise, as testing is performed within controlled parameter spaces. A wearable monitoring tool to be used beyond fitness and heart rate must account for noise sources originating from individual patient variations (e.g., skin tone, obesity, age, and gender), physiology (e.g., respiration, venous pulsation, body site of measurement, and body temperature), and external perturbations of the device itself (e.g., motion artifact, ambient light, and applied pressure to the skin). Here, we present a comprehensive review of the literature that aims to summarize these noise sources for future PPG device development for use in health monitoring