Digital phenotyping through multimodal, unobtrusive sensing

The growing adoption of multimodal wearable and mobile devices, such as smartphones and wrist-worn watches has generated an increase in the collection of physiological and behavioural data at scale. This digital phenotyping data enables researchers to make inferences regarding users’ physical and mental health at scale, for the first time. However, translating this data into actionable insights requires computational approaches that turn unlabelled, multimodal time-series sensor data into validated measures that can be interpreted at scale. This thesis describes the derivation of novel computational methods that leverage digital phenotyping data from wearable devices in large-scale populations to infer physical behaviours. These methods combine insights from signal processing, data mining and machine learning alongside domain knowledge in physical activity and sleep epidemiology. First, the inference of sleeping windows in free-living conditions through a heart rate sensing approach is explored. This algorithm is particularly valuable in the absence of ground truth or sleep diaries given its simplicity, adaptability and capacity for personalization. I then explore multistage sleep classification through combined movement and cardiac wearable sensing and machine learning. Further, I demonstrate that postural changes detected through wrist accelerometers can inform habitual behaviours and are valuable complements to traditional, intensity-based physical activity metrics. I then leverage the concomitant responses of heart rate to physical activity that can be captured through multimodal wearable sensors through a self-supervised training task. The resulting embeddings from this task are shown to be useful for the downstream classification of demographic factors, BMI, energy expenditure and cardiorespiratory fitness. Finally, I describe a deep learning model for the adaptive inference of cardiorespiratory fitness (VO2max) using wearable data in free living conditions. I demonstrate the robustness of the model in a large UK population and show the models’ adaptability by evaluating its performance in a subset of the population with repeated measures ~6 years after the original recordings. Together, this work increases the potential of multimodal wearable and mobile sensors for physical activity and behavioural inferences in population studies. In particular, this thesis showcases the potential of using wearable devices to make valuable physical activity, sleep and fitness inferences in large cohort studies. Given the nature of the data collected and the fact that most of this data is currently generated by commercial providers and not research institutes, laying the foundations for responsible data governance and ethical use of these technologies will be critical to building trust and enabling the development of the field of digital phenotyping.I was funded by GlaxoSmithKline and the Engineering and Physical Sciences Research Council. I was also supported by the Alan Turing Institute through their Enrichment Scheme

Sleep Stage Classification: A Deep Learning Approach

Sleep occupies significant part of human life. The diagnoses of sleep related disorders are of great importance. To record specific physical and electrical activities of the brain and body, a multi-parameter test, called polysomnography (PSG), is normally used. The visual process of sleep stage classification is time consuming, subjective and costly. To improve the accuracy and efficiency of the sleep stage classification, automatic classification algorithms were developed. In this research work, we focused on pre-processing (filtering boundaries and de-noising algorithms) and classification steps of automatic sleep stage classification. The main motivation for this work was to develop a pre-processing and classification framework to clean the input EEG signal without manipulating the original data thus enhancing the learning stage of deep learning classifiers. For pre-processing EEG signals, a lossless adaptive artefact removal method was proposed. Rather than other works that used artificial noise, we used real EEG data contaminated with EOG and EMG for evaluating the proposed method. The proposed adaptive algorithm led to a significant enhancement in the overall classification accuracy. In the classification area, we evaluated the performance of the most common sleep stage classifiers using a comprehensive set of features extracted from PSG signals. Considering the challenges and limitations of conventional methods, we proposed two deep learning-based methods for classification of sleep stages based on Stacked Sparse AutoEncoder (SSAE) and Convolutional Neural Network (CNN). The proposed methods performed more efficiently by eliminating the need for conventional feature selection and feature extraction steps respectively. Moreover, although our systems were trained with lower number of samples compared to the similar studies, they were able to achieve state of art accuracy and higher overall sensitivity

Sviluppo di un metodo innovativo per la misura del comfort termico attraverso il monitoraggio di parametri fisiologici e ambientali in ambienti indoor

openLa misura del comfort termico in ambienti indoor è un argomento di interesse per la comunità scientifica, poiché il comfort termico incide profondamente sul benessere degli utenti ed inoltre, per garantire condizioni di comfort ottimali, gli edifici devono affrontare costi energetici elevati. Anche se esistono norme nel campo dell'ergonomia del comfort che forniscono linee guida per la valutazione del comfort termico, può succedere che in contesti reali sia molto difficile ottenere una misurazione accurata. Pertanto, per migliorare la misura del comfort termico negli edifici, la ricerca si sta concentrando sulla valutazione dei parametri personali e fisiologici legati al comfort termico, per creare ambienti su misura per l’utente. Questa tesi presenta diversi contributi riguardo questo argomento. Infatti, in questo lavoro di ricerca, sono stati implementati una serie di studi per sviluppare e testare procedure di misurazione in grado di valutare quantitativamente il comfort termico umano, tramite parametri ambientali e fisiologici, per catturare le peculiarità che esistono tra i diversi utenti. In primo luogo, è stato condotto uno studio in una camera climatica controllata, con un set di sensori invasivi utilizzati per la misurazione dei parametri fisiologici. L'esito di questa ricerca è stato utile per ottenere una prima accuratezza nella misurazione del comfort termico dell'82%, ottenuta mediante algoritmi di machine learning (ML) che forniscono la sensazione termica (TSV) utilizzando la variabilità della frequenza cardiaca (HRV) , parametro che la letteratura ha spesso riportato legato sia al comfort termico dell'utenza che alle grandezze ambientali. Questa ricerca ha dato origine a uno studio successivo in cui la valutazione del comfort termico è stata effettuata utilizzando uno smartwatch minimamente invasivo per la raccolta dell’HRV. Questo secondo studio consisteva nel variare le condizioni ambientali di una stanza semi-controllata, mentre i partecipanti potevano svolgere attività di ufficio ma in modo limitato, ovvero evitando il più possibile i movimenti della mano su cui era indossato lo smartwatch. Con questa configurazione, è stato possibile stabilire che l'uso di algoritmi di intelligenza artificiale (AI) e il set di dati eterogeneo creato aggregando parametri ambientali e fisiologici, può fornire una misura di TSV con un errore medio assoluto (MAE) di 1.2 e un errore percentuale medio assoluto (MAPE) del 20%. Inoltre, tramite il Metodo Monte Carlo (MCM) è stato possibile calcolare l'impatto delle grandezze in ingresso sul calcolo del TSV. L'incertezza più alta è stata raggiunta a causa dell'incertezza nella misura della temperatura dell'aria (U = 14%) e dell'umidità relativa (U = 10,5%). L'ultimo contributo rilevante ottenuto con questa ricerca riguarda la misura del comfort termico in ambiente reale, semi controllato, in cui il partecipante non è stato costretto a limitare i propri movimenti. La temperatura della pelle è stata inclusa nel set-up sperimentale, per migliorare la misurazione del TSV. I risultati hanno mostrato che l'inclusione della temperatura della pelle per la creazione di modelli personalizzati, realizzati utilizzando i dati provenienti dal singolo partecipante, porta a risultati soddisfacenti (MAE = 0,001±0,0003 e MAPE = 0,02%±0,09%). L'approccio più generalizzato, invece, che consiste nell'addestrare gli algoritmi sull'intero gruppo di partecipanti tranne uno, e utilizzare quello tralasciato per il test, fornisce prestazioni leggermente inferiori (MAE = 1±0.2 e MAPE = 25% ±6%). Questo risultato evidenzia come in condizioni semi-controllate, la previsione di TSV utilizzando la temperatura della pelle e l'HRV possa essere eseguita con un certo grado di incertezza.Measuring human thermal comfort in indoor environments is a topic of interest in the scientific community, since thermal comfort deeply affects the well-being of occupants and furthermore, to guarantee optimal comfort conditions, buildings must face high energy costs. Even if there are standards in the field of the ergonomics of the thermal environment that provide guidelines for thermal comfort assessment, it can happen that in real-world settings it is very difficult to obtain an accurate measurement. Therefore, to improve the measurement of thermal comfort of occupants in buildings, research is focusing on the assessment of personal and physiological parameters related to thermal comfort, to create environments carefully tailored to the occupant that lives in it. This thesis presents several contributions to this topic. In fact, in the following research work, a set of studies were implemented to develop and test measurement procedures capable of quantitatively assessing human thermal comfort, by means of environmental and physiological parameters, to capture peculiarities among different occupants. Firstly, it was conducted a study in a controlled climatic chamber with an invasive set of sensors used for measuring physiological parameters. The outcome of this research was helpful to achieve a first accuracy in the measurement of thermal comfort of 82%, obtained by training machine learning (ML) algorithms that provide the thermal sensation vote (TSV) by means of environmental quantities and heart rate variability (HRV), a parameter that literature has often reported being related to both users' thermal comfort. This research gives rise to a subsequent study in which thermal comfort assessment was made by using a minimally invasive smartwatch for collecting HRV. This second study consisted in varying the environmental conditions of a semi-controlled test-room, while participants could carry out light-office activities but in a limited way, i.e. avoiding the movements of the hand on which the smartwatch was worn as much as possible. With this experimental setup, it was possible to establish that the use of artificial intelligence (AI) algorithms (such as random forest or convolutional neural networks) and the heterogeneous dataset created by aggregating environmental and physiological parameters, can provide a measure of TSV with a mean absolute error (MAE) of 1.2 and a mean absolute percentage error (MAPE) of 20%. In addition, by using of Monte Carlo Method (MCM), it was possible to compute the impact of the uncertainty of the input quantities on the computation of the TSV. The highest uncertainty was reached due to the air temperature uncertainty (U = 14%) and relative humidity (U = 10.5%). The last relevant contribution obtained with this research work concerns the measurement of thermal comfort in a real-life setting, semi-controlled environment, in which the participant was not forced to limit its movements. Skin temperature was included in the experimental set-up, to improve the measurement of TSV. The results showed that the inclusion of skin temperature for the creation of personalized models, made by using data coming from the single participant brings satisfactory results (MAE = 0.001±0.0003 and MAPE = 0.02%±0.09%). On the other hand, the more generalized approach, which consists in training the algorithms on the whole bunch of participants except one, and using the one left out for the test, provides slightly lower performances (MAE = 1±0.2 and MAPE = 25%±6%). This result highlights how in semi-controlled conditions, the prediction of TSV using skin temperature and HRV can be performed with acceptable accuracy.INGEGNERIA INDUSTRIALEembargoed_20220321Morresi, Nicol

Intelligent Biosignal Analysis Methods

This book describes recent efforts in improving intelligent systems for automatic biosignal analysis. It focuses on machine learning and deep learning methods used for classification of different organism states and disorders based on biomedical signals such as EEG, ECG, HRV, and others

Personalized data analytics for internet-of-things-based health monitoring

The Internet-of-Things (IoT) has great potential to fundamentally alter the delivery of modern healthcare, enabling healthcare solutions outside the limits of conventional clinical settings. It can offer ubiquitous monitoring to at-risk population groups and allow diagnostic care, preventive care, and early intervention in everyday life. These services can have profound impacts on many aspects of health and well-being. However, this field is still at an infancy stage, and the use of IoT-based systems in real-world healthcare applications introduces new challenges. Healthcare applications necessitate satisfactory quality attributes such as reliability and accuracy due to their mission-critical nature, while at the same time, IoT-based systems mostly operate over constrained shared sensing, communication, and computing resources. There is a need to investigate this synergy between the IoT technologies and healthcare applications from a user-centered perspective. Such a study should examine the role and requirements of IoT-based systems in real-world health monitoring applications. Moreover, conventional computing architecture and data analytic approaches introduced for IoT systems are insufficient when used to target health and well-being purposes, as they are unable to overcome the limitations of IoT systems while fulfilling the needs of healthcare applications. This thesis aims to address these issues by proposing an intelligent use of data and computing resources in IoT-based systems, which can lead to a high-level performance and satisfy the stringent requirements. For this purpose, this thesis first delves into the state-of-the-art IoT-enabled healthcare systems proposed for in-home and in-hospital monitoring. The findings are analyzed and categorized into different domains from a user-centered perspective. The selection of home-based applications is focused on the monitoring of the elderly who require more remote care and support compared to other groups of people. In contrast, the hospital-based applications include the role of existing IoT in patient monitoring and hospital management systems. Then, the objectives and requirements of each domain are investigated and discussed. This thesis proposes personalized data analytic approaches to fulfill the requirements and meet the objectives of IoT-based healthcare systems. In this regard, a new computing architecture is introduced, using computing resources in different layers of IoT to provide a high level of availability and accuracy for healthcare services. This architecture allows the hierarchical partitioning of machine learning algorithms in these systems and enables an adaptive system behavior with respect to the user's condition. In addition, personalized data fusion and modeling techniques are presented, exploiting multivariate and longitudinal data in IoT systems to improve the quality attributes of healthcare applications. First, a real-time missing data resilient decision-making technique is proposed for health monitoring systems. The technique tailors various data resources in IoT systems to accurately estimate health decisions despite missing data in the monitoring. Second, a personalized model is presented, enabling variations and event detection in long-term monitoring systems. The model evaluates the sleep quality of users according to their own historical data. Finally, the performance of the computing architecture and the techniques are evaluated in this thesis using two case studies. The first case study consists of real-time arrhythmia detection in electrocardiography signals collected from patients suffering from cardiovascular diseases. The second case study is continuous maternal health monitoring during pregnancy and postpartum. It includes a real human subject trial carried out with twenty pregnant women for seven months

Simulation and implementation of novel deep learning hardware architectures for resource constrained devices

Corey Lammie designed mixed signal memristive-complementary metal–oxide–semiconductor (CMOS) and field programmable gate arrays (FPGA) hardware architectures, which were used to reduce the power and resource requirements of Deep Learning (DL) systems; both during inference and training. Disruptive design methodologies, such as those explored in this thesis, can be used to facilitate the design of next-generation DL systems

Advanced Signal Processing in Wearable Sensors for Health Monitoring

Smart, wearables devices on a miniature scale are becoming increasingly widely available, typically in the form of smart watches and other connected devices. Consequently, devices to assist in measurements such as electroencephalography (EEG), electrocardiogram (ECG), electromyography (EMG), blood pressure (BP), photoplethysmography (PPG), heart rhythm, respiration rate, apnoea, and motion detection are becoming more available, and play a significant role in healthcare monitoring. The industry is placing great emphasis on making these devices and technologies available on smart devices such as phones and watches. Such measurements are clinically and scientifically useful for real-time monitoring, long-term care, and diagnosis and therapeutic techniques. However, a pertaining issue is that recorded data are usually noisy, contain many artefacts, and are affected by external factors such as movements and physical conditions. In order to obtain accurate and meaningful indicators, the signal has to be processed and conditioned such that the measurements are accurate and free from noise and disturbances. In this context, many researchers have utilized recent technological advances in wearable sensors and signal processing to develop smart and accurate wearable devices for clinical applications. The processing and analysis of physiological signals is a key issue for these smart wearable devices. Consequently, ongoing work in this field of study includes research on filtration, quality checking, signal transformation and decomposition, feature extraction and, most recently, machine learning-based methods