Multimodal Signal Processing for Diagnosis of Cardiorespiratory Disorders

This thesis addresses the use of multimodal signal processing to develop algorithms for the automated processing of two cardiorespiratory disorders. The aim of the first application of this thesis was to reduce false alarm rate in an intensive care unit. The goal was to detect five critical arrhythmias using processing of multimodal signals including photoplethysmography, arterial blood pressure, Lead II and augmented right arm electrocardiogram (ECG). A hierarchical approach was used to process the signals as well as a custom signal processing technique for each arrhythmia type. Sleep disorders are a prevalent health issue, currently costly and inconvenient to diagnose, as they normally require an overnight hospital stay by the patient. In the second application of this project, we designed automated signal processing algorithms for the diagnosis of sleep apnoea with a main focus on the ECG signal processing. We estimated the ECG-derived respiratory (EDR) signal using different methods: QRS-complex area, principal component analysis (PCA) and kernel PCA. We proposed two algorithms (segmented PCA and approximated PCA) for EDR estimation to enable applying the PCA method to overnight recordings and rectify the computational issues and memory requirement. We compared the EDR information against the chest respiratory effort signals. The performance was evaluated using three automated machine learning algorithms of linear discriminant analysis (LDA), extreme learning machine (ELM) and support vector machine (SVM) on two databases: the MIT PhysioNet database and the St. Vincent’s database. The results showed that the QRS area method for EDR estimation combined with the LDA classifier was the highest performing method and the EDR signals contain respiratory information useful for discriminating sleep apnoea. As a final step, heart rate variability (HRV) and cardiopulmonary coupling (CPC) features were extracted and combined with the EDR features and temporal optimisation techniques were applied. The cross-validation results of the minute-by-minute apnoea classification achieved an accuracy of 89%, a sensitivity of 90%, a specificity of 88%, and an AUC of 0.95 which is comparable to the best results reported in the literature

Factors affecting daytime function in the sleep apnoea/hypopnoea syndrome

The sleep apnoea/hypopnoea syndrome (SAHS) is characterised by repetitive upper airway obstructions during sleep, which lead to recurrent hypoxaemia and brief arousals from sleep. SAHS patients suffer from excessive daytime sleepiness (EDS), cognitive impairments and decreased psychological well -being. Previous studies have examined relationships between the nocturnal events of SAHS and a limited number of daytime function measures, frequently in small, non -consecutive patient samples. Relationships found have been either weak or non -significant. This thesis examines the relationships between a wide range of nocturnal sleep and breathing variables and daytime function. Additionally, this thesis examines the use of subjective and objective measures of daytime sleepiness, to determine which tests provide the most useful information for SAHS patients.A pilot study found that neither the 103 patients' nor their partners' Epworth rating of sleepiness were strong predictors of SAHS severity. In 150 patients with a wide range of SAHS severity, relationships between nocturnal events and daytime function were examined using newer definitions of arousal and measures of sleep continuity. A broad battery of daytime tests were used including the maintenance of wakefulness test (MWT) and the short form (SF) -36. Unlike previous studies, all correlations were controlled for age and awake oxygen saturation, known to influence the variables measured. The current study also examined these correlations in an unselected patient sample with a range of disease severity. The study found a lack of strong relationships between conventional nocturnal sleep and breathing variables and daytime function. Few baseline variables significantly predicted CPAP use.Daytime function measures were compared within the 150 patients. The multiple sleep latency test (MSLT) and the MWT displayed a moderate, discordant relationship. Measures of cognitive function, psychological well -being and subjective sleepiness ii better related to the MWT than MSLT, suggesting that the MWT may be a more useful tool in assessing functional impairment in sleep apnoea.A randomised cross -over study, on 12 SANS patients, compared daytime sleepiness measured following a night's sleep at home (as performed in this thesis) versus a night in the sleep centre (standard protocol). Preliminary results indicated that daytime sleepiness, as measured by the MSLT and MWT, was not significantly different between the two study limbs. This suggests that the non -standard method of conducting the MSLT and MWT in this thesis does not explain the lack of correlational relationships between nocturnal measures and daytime sleepiness.The studies presented in this thesis demonstrate a lack of identified factors affecting daytime function in a group of unselected SANS patients. This may be due to inter - individual patient variability. Also, more sophisticated nocturnal SANS measures should be examined, as should more `real -life' daytime assessments, such as ambulatory EEG recorded during a patient's normal daily routine

ARTIFICIAL INTELLIGENCE-ENABLED EDGE-CENTRIC SOLUTION FOR AUTOMATED ASSESSMENT OF SLEEP USING WEARABLES IN SMART HEALTH

ARTIFICIAL INTELLIGENCE-ENABLED EDGE-CENTRIC SOLUTION FOR AUTOMATED ASSESSMENT OF SLEEP USING WEARABLES IN SMART HEALT

Temporal contextual descriptors and applications to emotion analysis.

The current trends in technology suggest that the next generation of services and devices allows smarter customization and automatic context recognition. Computers learn the behavior of the users and can offer them customized services depending on the context, location, and preferences. One of the most important challenges in human-machine interaction is the proper understanding of human emotions by machines and automated systems. In the recent years, the progress made in machine learning and pattern recognition led to the development of algorithms that are able to learn the detection and identification of human emotions from experience. These algorithms use different modalities such as image, speech, and physiological signals to analyze and learn human emotions. In many settings, the vocal information might be more available than other modalities due to widespread of voice sensors in phones, cars, and computer systems in general. In emotion analysis from speech, an audio utterance is represented by an ordered (in time) sequence of features or a multivariate time series. Typically, the sequence is further mapped into a global descriptor representative of the entire utterance/sequence. This descriptor is used for classification and analysis. In classic approaches, statistics are computed over the entire sequence and used as a global descriptor. This often results in the loss of temporal ordering from the original sequence. Emotion is a succession of acoustic events. By discarding the temporal ordering of these events in the mapping, the classic approaches cannot detect acoustic patterns that lead to a certain emotion. In this dissertation, we propose a novel feature mapping framework. The proposed framework maps temporally ordered sequence of acoustic features into data-driven global descriptors that integrate the temporal information from the original sequence. The framework contains three mapping algorithms. These algorithms integrate the temporal information implicitly and explicitly in the descriptor\u27s representation. In the rst algorithm, the Temporal Averaging Algorithm, we average the data temporally using leaky integrators to produce a global descriptor that implicitly integrates the temporal information from the original sequence. In order to integrate the discrimination between classes in the mapping, we propose the Temporal Response Averaging Algorithm which combines the temporal averaging step of the previous algorithm and unsupervised learning to produce data driven temporal contextual descriptors. In the third algorithm, we use the topology preserving property of the Self-Organizing Maps and the continuous nature of speech to map a temporal sequence into an ordered trajectory representing the behavior over time of the input utterance on a 2-D map of emotions. The temporal information is integrated explicitly in the descriptor which makes it easier to monitor emotions in long speeches. The proposed mapping framework maps speech data of different length to the same equivalent representation which alleviates the problem of dealing with variable length temporal sequences. This is advantageous in real time setting where the size of the analysis window can be variable. Using the proposed feature mapping framework, we build a novel data-driven speech emotion detection and recognition system that indexes speech databases to facilitate the classification and retrieval of emotions. We test the proposed system using two datasets. The first corpus is acted. We showed that the proposed mapping framework outperforms the classic approaches while providing descriptors that are suitable for the analysis and visualization of humans’ emotions in speech data. The second corpus is an authentic dataset. In this dissertation, we evaluate the performances of our system using a collection of debates. For that purpose, we propose a novel debate collection that is one of the first initiatives in the literature. We show that the proposed system is able to learn human emotions from debates

Wearable in-ear pulse oximetry: theory and applications

Wearable health technology, most commonly in the form of the smart watch, is employed by millions of users worldwide. These devices generally exploit photoplethysmography (PPG), the non-invasive use of light to measure blood volume, in order to track physiological metrics such as pulse and respiration. Moreover, PPG is commonly used in hospitals in the form of pulse oximetry, which measures light absorbance by the blood at different wavelengths of light to estimate blood oxygen levels (SpO2). This thesis aims to demonstrate that despite its widespread usage over many decades, this sensor still possesses a wealth of untapped value. Through a combination of advanced signal processing and harnessing the ear as a location for wearable sensing, this thesis introduces several novel high impact applications of in-ear pulse oximetry and photoplethysmography. The aims of this thesis are accomplished through a three pronged approach: rapid detection of hypoxia, tracking of cognitive workload and fatigue, and detection of respiratory disease. By means of the simultaneous recording of in-ear and finger pulse oximetry at rest and during breath hold tests, it was found that in-ear SpO2 responds on average 12.4 seconds faster than the finger SpO2. This is likely due in part to the ear being in close proximity to the brain, making it a priority for oxygenation and thus making wearable in-ear SpO2 a good proxy for core blood oxygen. Next, the low latency of in-ear SpO2 was further exploited in the novel application of classifying cognitive workload. It was found that in-ear pulse oximetry was able to robustly detect tiny decreases in blood oxygen during increased cognitive workload, likely caused by increased brain metabolism. This thesis demonstrates that in-ear SpO2 can be used to accurately distinguish between different levels of an N-back memory task, representing different levels of mental effort. This concept was further validated through its application to gaming and then extended to the detection of driver related fatigue. It was found that features derived from SpO2 and PPG were predictive of absolute steering wheel angle, which acts as a proxy for fatigue. The strength of in-ear PPG for the monitoring of respiration was investigated with respect to the finger, with the conclusion that in-ear PPG exhibits far stronger respiration induced intensity variations and pulse amplitude variations than the finger. All three respiratory modes were harnessed through multivariate empirical mode decomposition (MEMD) to produce spirometry-like respiratory waveforms from PPG. It was discovered that these PPG derived respiratory waveforms can be used to detect obstruction to breathing, both through a novel apparatus for the simulation of breathing disorders and through the classification of chronic obstructive pulmonary disease (COPD) in the real world. This thesis establishes in-ear pulse oximetry as a wearable technology with the potential for immense societal impact, with applications from the classification of cognitive workload and the prediction of driver fatigue, through to the detection of chronic obstructive pulmonary disease. The experiments and analysis in this thesis conclusively demonstrate that widely used pulse oximetry and photoplethysmography possess a wealth of untapped value, in essence teaching the old PPG sensor new tricks.Open Acces

Neuromarketing and global branding reaction analysis based on real-time monitoring of multiple consumer's biosignals and emotions

Consumers' selections and decision-making processes are some of the most exciting and challenging topics in neuromarketing, sales, and branding. From a global perspective, multicultural influences and societal conditions are crucial to consider. Neuroscience applications in international marketing and consumer behavior is an emergent and multidisciplinary field aiming to understand consumers' thoughts, reactions, and selection processes in branding and sales. This study focuses on real-time monitoring of different physiological signals using eye-tracking, facial expressions recognition, and Galvanic Skin Response (GSR) acquisition methods to analyze consumers' responses, detect emotional arousal, measure attention or relaxation levels, analyze perception, consciousness, memory, learning, motivation, preference, and decision-making. This research aimed to monitor human subjects' reactions to these signals during an experiment designed in three phases consisting of different branding advertisements. The nonadvertisement exposition was also monitored while gathering survey responses at the end of each phase. A feature extraction module with a data analytics module was implemented to calculate statistical metrics and decision-making supporting tools based on Principal Component Analysis (PCA) and Feature Importance (FI) determination based on the Random Forest technique. The results indicate that when compared to image ads, video ads are more effective in attracting consumers' attention and creating more emotional arousal.https://doi.org/10.37227/JIBM-2023-04-5912Published versio

The Neural Mechanisms of Sleep and Migraine

Whilst a bidirectional relationship between sleep and migraine has long been postulated, this remains mainly speculative, and the underlying neural mechanisms remain to be determined. In this thesis we sought to explore this with clinical and preclinical methodologies. It was hypothesised that disrupted sleep-wake and nociception-regulating neural networks including key brainstem and diencephalic structures alter the thresholds for attack initiation and increase migraine susceptibility.Firstly, we used a meta-analytic approach to determine whether migraine patients have altered sleep, identifying that they have poorer subjective sleep quality and altered sleep physiology including reduced rapid-eye-movement sleep, compared to healthy controls. By collating data from users of the Migraine Buddy application (Healint Ltd.) and conducting Bayesian regression models we explored whether changes in sleep were predictors of an attack and conversely whether experiencing an attack would predict changes in subsequent sleep. We determined that interrupted sleep and deviations from typical sleep were potential predictors of a next day migraine attack but having an attack did not predict sleep duration.Secondly, we utilised mouse models of sleep deprivation and demonstrated that this led to orofacial mechanical allodynia - a commonly reported migraine phenotype indicative of sensitisation of the trigeminovascular system. Mechanistic insight was provided in that orexin-A, a hypothalamic arousal-promoting peptide which stabilises sleep-wake transitions reversed this phenotype.Finally, we explored whether familial natural short sleepers (FNSS) which are reported to have increased orexin expression, are less susceptible to migraine-related phenotypes using a transgenic mouse line harbouring the P384R mutation in the hDEC2 gene. We observed no significant differences in migraine-related phenotypes at baseline, however, when exposed to a clinical migraine trigger (nitroglycerin) FNSS mice demonstrated reduced orofacial hypersensitivity and photophobia, indicative of decreased migraine susceptibility. FNSS also displayed alterations in metabolites underlying energy metabolism and oxidative stress, suggesting a potential link between metabolism and headache pathophysiology.Taken together, the data in this thesis has shed light on the relationship between sleep and migraine, highlighting alterations in sleep as a potential precipitant of migraine attacks, and identifying genetic mechanisms underlying sleep regulation which may curtail migraine development, as well as possible therapeutic targets based on the orexinergic system. Although further work is needed to fully understand this neural basis, this has promising clinical implications and has furthered our understanding of migraine pathophysiology.<br/

Computational Sleep Science: Machine Learning for the Detection, Diagnosis, and Treatment of Sleep Problems from Wearable Device Data

University of Minnesota Ph.D. dissertation.December 2017. Major: Computer Science. Advisor: Jaideep Srivastava. 1 computer file (PDF); xiii, 122 pages.This thesis is motivated by the rapid increase in global life expectancy without the respective improvements in quality of life. I propose several novel machine learning and data mining methodologies for approaching a paramount component of quality of life, the translational science field of sleep research. Inadequate sleep negatively affects both mental and physical well-being, and exacerbates many non-communicable health problems such as diabetes, depression, cancer and obesity. Taking advantage of the ubiquitous adoption of wearable devices, I create algorithmic solutions to analyse sensor data. The goal is to improve the quality of life of wearable device users, as well as provide clinical insights and tools for sleep researchers and care-providers. Chapter 1 is the introduction. This section substantiates the timely relevance of sleep research for today's society, and its contribution towards improved global health. It covers the history of sleep science technology and identifies core computing challenges in the field. The scope of the thesis is established and an approach is articulated. Useful definitions, sleep domain terminology, and some pre-processing steps are defined. Lastly, an outline for the remainder of the thesis is included. Chapter 2 dives into my proposed methodology for widespread screening of sleep disorders. It surveys results from the application of several statistical and data mining methods. It also introduces my novel deep learning architecture optimized for the unique dimensionality and nature of wearable device data. Chapter 3 focuses on the diagnosis stage of the sleep science process. I introduce a human activity recognition algorithm called RAHAR, Robust Automated Human Activity Recognition. This algorithm is unique in a number of ways, including its objective of annotating a behavioural time series with exertion levels rather than activity type. Chapter 4 focuses on the last step of the sleep science process, therapy. I define a pipeline to identify \textit{behavioural recipes}. These \textit{recipes} are the target behaviour that a user should complete in order to have good quality sleep. This work provides the foundation for building out a dynamic real-time recommender system for wearable device users, or a clinically administered cognitive behavioural therapy program. Chapter 5 summarizes the impact of this body of work, and takes a look into next steps. This chapter concludes my thesis