Neonatal non-contact respiratory monitoring based on real-time infrared thermography

<p>Abstract</p> <p>Background</p> <p>Monitoring of vital parameters is an important topic in neonatal daily care. Progress in computational intelligence and medical sensors has facilitated the development of smart bedside monitors that can integrate multiple parameters into a single monitoring system. This paper describes non-contact monitoring of neonatal vital signals based on infrared thermography as a new biomedical engineering application. One signal of clinical interest is the spontaneous respiration rate of the neonate. It will be shown that the respiration rate of neonates can be monitored based on analysis of the anterior naris (nostrils) temperature profile associated with the inspiration and expiration phases successively.</p> <p>Objective</p> <p>The aim of this study is to develop and investigate a new non-contact respiration monitoring modality for neonatal intensive care unit (NICU) using infrared thermography imaging. This development includes subsequent image processing (region of interest (ROI) detection) and optimization. Moreover, it includes further optimization of this non-contact respiration monitoring to be considered as physiological measurement inside NICU wards.</p> <p>Results</p> <p>Continuous wavelet transformation based on Debauches wavelet function was applied to detect the breathing signal within an image stream. Respiration was successfully monitored based on a 0.3°C to 0.5°C temperature difference between the inspiration and expiration phases.</p> <p>Conclusions</p> <p>Although this method has been applied to adults before, this is the first time it was used in a newborn infant population inside the neonatal intensive care unit (NICU). The promising results suggest to include this technology into advanced NICU monitors.</p

Infrared thermography and image analysis for biomedical use

Infrared thermography is used for measuring and analyzing physiological functions and pathology related to the body’s thermal homeostasis and temperature. This review provides an overview of the technological advantages of infrared imaging, with the focus on new advances in and opportunities for infrared imaging, as a reliable medical diagnostic tool. The review has four main parts. Firstly, a short history of thermography development in medicine is given. Secondly, an overview on the clinical and biomedical research results and methodological improvements in established applications of infrared thermography is provided. Thirdly, the details of published research and development results and activities of the last 3 years for time and frequency domain analysis of infrared video thermography recordings to study some vital functions of human physiology are discussed. Analysis of infrared video thermography streams resulted in important information on microvascular (arteriolar) function of the skin and of vital organs when exposed during an operation. This new set of parameters of microvascular function enhances the assessment of the cardiovascular system in chronic diseases e.g. in hypertension and diabetes. Infrared thermography provides valuable information when an organ’s suitability for transplantationmust be assessed based on quantifiable parameters of organ function and viability. Fourthly, a brief overview on a separate, exciting area of infrared imaging is provided as well: the development of a touchless polygraph system. It enables the study of the psychophysiological parameters of stress, by the assessment of breathing and pulse wave patterns by noncontact methodology, for lie detection purposes. In conclusion, infrared imaging is a non-invasive, non-radiative, low cost detection tool, and its application area is constantly growing, along with technical improvements and advances

Assessment of Non-Invasive Blood Pressure Prediction from PPG and rPPG Signals Using Deep Learning

Exploiting photoplethysmography signals (PPG) for non-invasive blood pressure (BP) measurement is interesting for various reasons. First, PPG can easily be measured using fingerclip sensors. Second, camera based approaches allow to derive remote PPG (rPPG) signals similar to PPG and therefore provide the opportunity for non-invasive measurements of BP. Various methods relying on machine learning techniques have recently been published. Performances are often reported as the mean average error (MAE) on the data which is problematic. This work aims to analyze the PPG- and rPPG based BP prediction error with respect to the underlying data distribution. First, we train established neural network (NN) architectures and derive an appropriate parameterization of input segments drawn from continuous PPG signals. Second, we use this parameterization to train NNs with a larger PPG dataset and carry out a systematic evaluation of the predicted blood pressure. The analysis revealed a strong systematic increase of the prediction error towards less frequent BP values across NN architectures. Moreover, we tested different train/test set split configurations which underpin the importance of a careful subject-aware dataset assignment to prevent overly optimistic results. Third, we use transfer learning to train the NNs for rPPG based BP prediction. The resulting performances are similar to the PPG-only case. Finally, we apply different personalization techniques and retrain our NNs with subject-specific data for both the PPG-only and rPPG case. Whilst the particular technique is less important, personalization reduces the prediction errors significantly

Examining the sense of agency in human-computer interaction

Humans are agents, we feel that we control the course of events on our everyday life. This refers to the Sense of Agency (SoA). This experience is not only crucial in our daily life, but also in our interaction with technology. When we manipulate a user interface (e.g., computer, smartphone, etc.), we expect that the system responds to our input commands with feedback, as we desire to feel that we are in charge of the interaction. If this interplay elicits a SoA, then the user will perceive an instinctive feeling of “I am controlling this”. Although research in Human-Computer Interaction (HCI) pursuits the design of intuitive and responsive systems, most of the current studies have been focussed mainly on interaction techniques (e.g., software-hardware) and User Experience (UX) (e.g., comfort, usability, etc.), and very little has been investigated in terms of the SoA i.e., the conscious experience of being in control regarding the interaction. In this thesis, we present an experimental exploration of the role of the SoA in interaction paradigms typical of HCI. After two chapters of introduction and related work, we describe a series of studies that explore agency implication in interaction with systems through human senses such as vision, audio, touch and smell. Chapter 3 explores the SoA in mid-air haptic interaction through touchless actions. Then, Chapter 4 examines agency modulation through smell and its application for olfactory interfaces. Chapter 5 describes two novel timing techniques based on auditory and haptic cues that provide alternative timing methods to the traditional Libet clock. Finally, we conclude with a discussion chapter that highlights the importance of our SoA during interactions with technology as well as the implications of the results found, in the design of user interfaces

A gaze-contingent framework for perceptually-enabled applications in healthcare

Patient safety and quality of care remain the focus of the smart operating room of the future. Some of the most influential factors with a detrimental effect are related to suboptimal communication among the staff, poor flow of information, staff workload and fatigue, ergonomics and sterility in the operating room. While technological developments constantly transform the operating room layout and the interaction between surgical staff and machinery, a vast array of opportunities arise for the design of systems and approaches, that can enhance patient safety and improve workflow and efficiency. The aim of this research is to develop a real-time gaze-contingent framework towards a "smart" operating suite, that will enhance operator's ergonomics by allowing perceptually-enabled, touchless and natural interaction with the environment. The main feature of the proposed framework is the ability to acquire and utilise the plethora of information provided by the human visual system to allow touchless interaction with medical devices in the operating room. In this thesis, a gaze-guided robotic scrub nurse, a gaze-controlled robotised flexible endoscope and a gaze-guided assistive robotic system are proposed. Firstly, the gaze-guided robotic scrub nurse is presented; surgical teams performed a simulated surgical task with the assistance of a robot scrub nurse, which complements the human scrub nurse in delivery of surgical instruments, following gaze selection by the surgeon. Then, the gaze-controlled robotised flexible endoscope is introduced; experienced endoscopists and novice users performed a simulated examination of the upper gastrointestinal tract using predominately their natural gaze. Finally, a gaze-guided assistive robotic system is presented, which aims to facilitate activities of daily living. The results of this work provide valuable insights into the feasibility of integrating the developed gaze-contingent framework into clinical practice without significant workflow disruptions.Open Acces

Beat-to-beat blood pressure estimation by photoplethysmography and its interpretation

Blood pressure (BP) is among the most important vital signals. Estimation of absolute BP solely using photoplethysmography (PPG) has gained immense attention over the last years. Available works differ in terms of used features as well as classifiers and bear large differences in their results. This work aims to provide a machine learning method for absolute BP estimation, its interpretation using computational methods and its critical appraisal in face of the current literature. We used data from three different sources including 273 subjects and 259,986 single beats. We extracted multiple features from PPG signals and its derivatives. BP was estimated by xgboost regression. For interpretation we used Shapley additive values (SHAP). Absolute systolic BP estimation using a strict separation of subjects yielded a mean absolute error of 9.456mmHg and correlation of 0.730. The results markedly improve if data separation is changed (MAE: 6.366mmHg, r: 0.874). Interpretation by means of SHAP revealed four features from PPG, its derivation and its decomposition to be most relevant. The presented approach depicts a general way to interpret multivariate prediction algorithms and reveals certain features to be valuable for absolute BP estimation. Our work underlines the considerable impact of data selection and of training/testing separation, which must be considered in detail when algorithms are to be compared. In order to make our work traceable, we have made all methods available to the public

VOLUNTARY CONTROL OF BREATHING ACCORDING TO THE BREATHING PATTERN DURING LISTENING TO MUSIC AND NON-CONTACT MEASUREMENT OF HEART RATE AND RESPIRATION

We investigated if listening to songs changes breathing pattern which changes autonomic responses such as heart rate (HR) and heart rate variability (HRV) or change in breathing pattern is a byproduct of listening to songs or change in breathing pattern as well as listening to songs causes changes in autonomic responses. Seven subjects (4 males and 3 females) participated in a pilot study where they listened to two types of songs and used a custom developed biofeedback program to control their breathing pattern to match the one recorded during listening to the songs. Coherencies between EEG, breathing pattern and RR intervals (RRI) were calculated to study the interaction with neural responses. Trends in HRV varied only during listening to songs, suggesting that autonomic response was affected by listening to songs irrespective of control of breathing. Effective coherence during songs while spontaneously breathing was more than during silence and during control of breathing. These results, although preliminary, suggest that listening to songs as well as change in breathing patterns changes the autonomic response but the effect of listening to songs may surpass the effect of changes in breathing. We explored feasibility of using non-contact measurements of HR and breathing rate (BR) by using recently developed Facemesh and other methods for tracking regions of interests from videos of faces of subjects. Performance was better for BR than HR, and over currently used methods. However, refinement of the approach would be needed to get the precision required for detecting subtle changes

Substance-specific modulation of the affective and neurobiological effects of heroin and cocaine in human addicts

This dissertation investigates how the settings of drug use influence the affective and neurobiological response to heroin versus cocaine in addicts. Chapter 1 reviews the neuropharmacology of heroin and cocaine and the theoretical background for drugs-settings interactions, including a detailed discussion of findings from previous studies in animals and humans that show how the same settings can influence in opposite directions the reinforcing effect of heroin and cocaine. Cocaine self-administration, for example, was greatly facilitated when rats were tested outside the home environment relative to rats test at home. The opposite pattern was found for heroin. Translational studies in humans yielded similar results. Indeed, heroin and cocaine co-abusers reported using the two drugs in distinct settings: heroin preferentially at home and cocaine preferentially outside the home. The aim of this dissertation is to determine whether the setting could also influence in opposite manner the affective and neurobiological response to heroin and cocaine in human addicts. Chapter 2 illustrates the findings of a study aimed at testing the hypothesis that the affective state experienced under cocaine or heroin is the result of an interaction between central and peripheral drug effects and the surroundings of drug use. According to this hypothesis, when cocaine is taken at home there is a mismatch between the familiar environment and the peripheral effects such as arousal, increased heart rate, increased respiratory rate, and increased muscular tension (which are usually produced in stressful situations). This mismatch dampens cocaine-rewarding effects. A mismatch would also occurs when heroin (which produces sedation and decreases heart rate, respiratory rate, and muscular tension) is used outside the home in contexts requiring vigilance. We found indeed that co-abusers subjectively experienced opposite changes in arousal, heart rate, respiratory rate, and muscular tension in response to cocaine (increase) versus heroin (decrease). Most important, using a novel two-dimensional visual test, we found that in agreement with the working hypothesis the valence of the affective state produced by heroin and cocaine shifted in opposite directions as a function of the setting of drug use: heroin was reported to be more pleasant at home than outside the home, and vice versa for cocaine. Chapter 3 illustrates the results of in which emotional imagery was combined with fMRI to investigation the neurobiological underpinnings of drug and setting interactions in addicts. Heroin and cocaine co-abusers were asked to recreate real-world settings of drug use during fMRI. In agreement with the working hypothesis, we found that heroin and cocaine imagery produced opposite changes in BOLD in the prefrontal cortex and in the striatum, regions implicated in brain reward in humans. Furthermore the same pattern of dissociation was observed in the cerebellum, suggesting that that a fronto-triatal-cerebellar network is implicated in processing drug-setting interactions. Chapter 4 includes a summary of the results, a general discussion, and suggestions for future research and implication. The major finding is that the environment surrounding drug use can influence in opposite manner the affective and neurobiological response to heroin and cocaine, suggesting that therapeutic approaches to the treatment of drug addiction should take into account the distinctive effects of different classes of drugs as well as the contexts of drug use. The Appendix includes reprints of two papers reporting on additional studies conducted during the course of the Ph.D. program, which are not directly germane to the aims of the dissertation. Other three papers are in the pre-submission stage

Biometric Spoofing: A JRC Case Study in 3D Face Recognition

Based on newly available and affordable off-the-shelf 3D sensing, processing and printing technologies, the JRC has conducted a comprehensive study on the feasibility of spoofing 3D and 2.5D face recognition systems with low-cost self-manufactured models and presents in this report a systematic and rigorous evaluation of the real risk posed by such attacking approach which has been complemented by a test campaign. The work accomplished and presented in this report, covers theories, methodologies, state of the art techniques, evaluation databases and also aims at providing an outlook into the future of this extremely active field of research.JRC.G.6-Digital Citizen Securit

IoT Platform for COVID-19 Prevention and Control: A Survey

As a result of the worldwide transmission of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), coronavirus disease 2019 (COVID-19) has evolved into an unprecedented pandemic. Currently, with unavailable pharmaceutical treatments and vaccines, this novel coronavirus results in a great impact on public health, human society, and global economy, which is likely to last for many years. One of the lessons learned from the COVID-19 pandemic is that a long-term system with non-pharmaceutical interventions for preventing and controlling new infectious diseases is desirable to be implemented. Internet of things (IoT) platform is preferred to be utilized to achieve this goal, due to its ubiquitous sensing ability and seamless connectivity. IoT technology is changing our lives through smart healthcare, smart home, and smart city, which aims to build a more convenient and intelligent community. This paper presents how the IoT could be incorporated into the epidemic prevention and control system. Specifically, we demonstrate a potential fog-cloud combined IoT platform that can be used in the systematic and intelligent COVID-19 prevention and control, which involves five interventions including COVID-19 Symptom Diagnosis, Quarantine Monitoring, Contact Tracing & Social Distancing, COVID-19 Outbreak Forecasting, and SARS-CoV-2 Mutation Tracking. We investigate and review the state-of-the-art literatures of these five interventions to present the capabilities of IoT in countering against the current COVID-19 pandemic or future infectious disease epidemics.Comment: 12 pages; Submitted to IEEE Internet of Things Journa