Digital phenotyping through multimodal, unobtrusive sensing

The growing adoption of multimodal wearable and mobile devices, such as smartphones and wrist-worn watches has generated an increase in the collection of physiological and behavioural data at scale. This digital phenotyping data enables researchers to make inferences regarding users’ physical and mental health at scale, for the first time. However, translating this data into actionable insights requires computational approaches that turn unlabelled, multimodal time-series sensor data into validated measures that can be interpreted at scale. This thesis describes the derivation of novel computational methods that leverage digital phenotyping data from wearable devices in large-scale populations to infer physical behaviours. These methods combine insights from signal processing, data mining and machine learning alongside domain knowledge in physical activity and sleep epidemiology. First, the inference of sleeping windows in free-living conditions through a heart rate sensing approach is explored. This algorithm is particularly valuable in the absence of ground truth or sleep diaries given its simplicity, adaptability and capacity for personalization. I then explore multistage sleep classification through combined movement and cardiac wearable sensing and machine learning. Further, I demonstrate that postural changes detected through wrist accelerometers can inform habitual behaviours and are valuable complements to traditional, intensity-based physical activity metrics. I then leverage the concomitant responses of heart rate to physical activity that can be captured through multimodal wearable sensors through a self-supervised training task. The resulting embeddings from this task are shown to be useful for the downstream classification of demographic factors, BMI, energy expenditure and cardiorespiratory fitness. Finally, I describe a deep learning model for the adaptive inference of cardiorespiratory fitness (VO2max) using wearable data in free living conditions. I demonstrate the robustness of the model in a large UK population and show the models’ adaptability by evaluating its performance in a subset of the population with repeated measures ~6 years after the original recordings. Together, this work increases the potential of multimodal wearable and mobile sensors for physical activity and behavioural inferences in population studies. In particular, this thesis showcases the potential of using wearable devices to make valuable physical activity, sleep and fitness inferences in large cohort studies. Given the nature of the data collected and the fact that most of this data is currently generated by commercial providers and not research institutes, laying the foundations for responsible data governance and ethical use of these technologies will be critical to building trust and enabling the development of the field of digital phenotyping.I was funded by GlaxoSmithKline and the Engineering and Physical Sciences Research Council. I was also supported by the Alan Turing Institute through their Enrichment Scheme

Vauvojen unen luokittelu patja-sensorilla ja EKG:lla

Infants spend the majority of their time asleep. Although extensive studies have been carried out, the role of sleep for infant cognitive, psychomotor, temperament and developmental outcomes is not clear. The current contradictory results may be due to the limited precision when monitoring infant sleep for prolonged periods of time, from weeks to even months. Sleep-wake cycle can be assessed with sleep questionnaires and actigraphy, but they cannot separate sleep stages. The gold standard for sleep state annotation is polysomnography (PSG), which consist of several signal modalities such as electroencephalogram, electrooculogram, electrocardiogram (ECG), electromyogram, respiration sensor and pulse oximetry. A sleep clinician manually assigns sleep stages for 30 sec epochs based on the visual observation of these signals. Because method is obtrusive and laborious it is not suitable for monitoring long periods. There is, therefore, a need for an automatic and unobtrusive sleep staging approach. In this work, a set of classifiers for infant sleep staging was created and evaluated. The cardiorespiratory and gross body movement signals were used as an input. The different classifiers aim to distinguish between two or more different sleep states. The classifiers were built on a clinical sleep polysomnography data set of 48 infants with ages ranging from 1 week to 18 weeks old (a median of 5 weeks). Respiration and gross body movements were observed using an electromechanical film bed mattress sensor manufactured by Emfit Ltd. ECG of the PSG setup was used for extracting cardiac activity. Signals were preprocessed to remove artefacts and an extensive set of features (N=81) were extracted on which the classifiers were trained. The NREM3 vs other states classifier provided the most accurate results. The median accuracy was 0.822 (IQR: 0.724-0.914). This is comparable to previously published studies on other sleep classifiers, as well as to the level of clinical interrater agreement. Classification methods were confounded by the lack of muscle atonia and amount of gross body movements in REM sleep. The proposed method could be readily applied for home monitoring, as well as for monitoring in neonatal intensive care units.Vauvat nukkuvat suurimman osan vuorokaudesta. Vaikkakin laajasti on tutkittu unen vaikutusta lapsen kognitioon, psykomotoriikkaan, temperamenttiin ja kehitykseen, selkeää kuvaa ja yhtenäistä konsensusta tiedeyhteisössä ei ole saavutettu. Yksi syy tähän on että ei ole olemassa menetelmää, joka soveltuisi jatkuva-aikaiseen ja pitkäkestoiseen unitilan monitorointiin. Vauvojen uni-valve- sykliä voidaan selvittää vanhemmille suunnatuilla kyselyillä ja aktigrafialla, mutta näillä ei voi havaita unitilojen rakennetta. Kliinisenä standardina unitilojen seurannassa on polysomnografia, jossa samanaikaisesti mitataan mm. potilaan elektroenkelografiaa, elektro-okulografiaa, elektrokardiografiaa, electromyografiaa, hengitysinduktiivisesta pletysmografiaa, happisaturaatiota ja hengitysvirtauksia. Kliinikko suorittaa univaiheluokittelun signaaleista näkyvien, vaiheille tyypillisten, hahmojen perusteella. Työläyden ja häiritsevän mittausasetelman takia menetelmä ei sovellu pitkäaikaiseen seurantaan. On tarvetta kehittää tarkoitukseen sopivia automaattisia ja huomaamattomia unenseurantamenetelmiä. Tässä työssä kehitettiin ja testattiin sydämen syke-, hengitys ja liikeanalyysiin perustuvia koneluokittimia vauvojen unitilojen havainnointiin. Luokittimet opetettiin kliinisessa polysomnografiassa kerätyllä datalla 48 vauvasta, joiden ikä vaihteli 1. viikosta 18. viikkoon (mediaani 5 viikkoa). Vauvojen hengitystä ja liikkeitä seurattiin Emfit Oy:n valmistamalla elektromekaaniseen filmiin pohjatuvalla patja-sensorilla. Lisäksi ECG:lla seurattiin sydäntä ja opetuksessa käytettiin lääkärin suorittamaa PSG-pohjaista luokitusta. Esikäsittelyn jälkeen signaaleista laskettiin suuri joukko piirrevektoreita (N=81), joihin luokittelu perustuu. NREM3-univaiheen tunnistus onnistui parhaiten 0.822 mediaani-tarkkuudella ja [0.724,0.914] kvartaaleilla. Tulos on yhtenevä kirjallisuudessa esitettyjen arvojen kanssa ja vastaa kliinikkojen välistä toistettavuutta. Muilla luokittimilla univaiheet sekoituivat keskenään, mikä on oletattavasti selitettävissä aikuisista poikeavalla REM-unen aikaisella lihasjäykkyydellä ja kehon liikkeillä. Työ osoittaa, että menetelmällä voi seurata vauvojen uniluokkien oskillaatiota. Järjestelmää voisi käyttää kotiseurannassa tai vastasyntyneiden teholla unenvalvontaan

Smart Sensors for Healthcare and Medical Applications

This book focuses on new sensing technologies, measurement techniques, and their applications in medicine and healthcare. Specifically, the book briefly describes the potential of smart sensors in the aforementioned applications, collecting 24 articles selected and published in the Special Issue “Smart Sensors for Healthcare and Medical Applications”. We proposed this topic, being aware of the pivotal role that smart sensors can play in the improvement of healthcare services in both acute and chronic conditions as well as in prevention for a healthy life and active aging. The articles selected in this book cover a variety of topics related to the design, validation, and application of smart sensors to healthcare

Non-Contact Sleep Monitoring

"The road ahead for preventive medicine seems clear. It is the delivery of high quality, personalised (as opposed to depersonalised) comprehensive medical care to all." Burney, Steiger, and Georges (1964) This world's population is ageing, and this is set to intensify over the next forty years. This demographic shift will result in signicant economic and societal burdens (partic- ularly on healthcare systems). The instantiation of a proactive, preventative approach to delivering healthcare is long recognised, yet is still proving challenging. Recent work has focussed on enabling older adults to age in place in their own homes. This may be realised through the recent technological advancements of aordable healthcare sen- sors and systems which continuously support independent living, particularly through longitudinally monitoring deviations in behavioural and health metrics. Overall health status is contingent on multiple factors including, but not limited to, physical health, mental health, and social and emotional wellbeing; sleep is implicitly linked to each of these factors. This thesis focusses on the investigation and development of an unobtrusive sleep mon- itoring system, particularly suited towards long-term placement in the homes of older adults. The Under Mattress Bed Sensor (UMBS) is an unobstrusive, pressure sensing grid designed to infer bed times and bed exits, and also for the detection of development of bedsores. This work extends the capacity of this sensor. Specically, the novel contri- butions contained within this thesis focus on an in-depth review of the state-of-the-art advances in sleep monitoring, and the development and validation of algorithms which extract and quantify UMBS-derived sleep metrics. Preliminary experimental and community deployments investigated the suitability of the sensor for long-term monitoring. Rigorous experimental development rened algorithms which extract respiration rate as well as motion metrics which outperform traditional forms of ambulatory sleep monitoring. Spatial, temporal, statistical and spatiotemporal features were derived from UMBS data as a means of describing movement during sleep. These features were compared across experimental, domestic and clinical data sets, and across multiple sleeping episodes. Lastly, the optimal classier (built using a combina- tion of the UMBS-derived features) was shown to infer sleep/wake state accurately and reliably across both younger and older cohorts. Through long-term deployment, it is envisaged that the UMBS-derived features (in- cluding spatial, temporal, statistical and spatiotemporal features, respiration rate, and sleep/wake state) may be used to provide unobtrusive, continuous insights into over- all health status, the progression of the symptoms of chronic conditions, and allow the objective measurement of daily (sleep/wake) patterns and routines

Models and Analysis of Vocal Emissions for Biomedical Applications

The MAVEBA Workshop proceedings, held on a biannual basis, collect the scientific papers presented both as oral and poster contributions, during the conference. The main subjects are: development of theoretical and mechanical models as an aid to the study of main phonatory dysfunctions, as well as the biomedical engineering methods for the analysis of voice signals and images, as a support to clinical diagnosis and classification of vocal pathologies

Early postnatal development of neocortex-wide activity patterns in GABAergic and pyramidal neurons

Before the onset of sensory experience, developing circuits generate synchronised activity that will not only influence its wiring, but ultimately contribute to behaviour. These complex functions rely on widely distributed cortical that simultaneously operate at multiple spatiotemporal scales. The timing of GABAergic maturation appears to align with the developmental trajectories of cortical regions, playing a crucial role in the functional development of individual brain areas. While local connectivity in cortical microcircuits has been extensively studied, the dynamics of brain-wide functional maturation, especially for GABAergic populations, remain underexplored. In this project, a dual-colour widefield calcium imaging approach was developed to examine the neocortex-wide dynamics of cortical GABAergic and excitatory neurons simultaneously across early postnatal development. This study provides the first broad description of neocortex-wide GABAergic developmental trajectories and their cross-talk with excitatory dynamics during the second and third postnatal weeks. The observed spontaneous activity revealed discrete activity domains, reflecting the modular organisation of the cortex. Both excitatory and GABAergic population exhibited an increase in the size and frequency of activity motifs, as well as changes in motif variability. However, as they matured, the distribution of these spatiotemporal properties displayed divergent trajectories across populations and regions. These findings suggest fundamental differences in the spatial organisation of both populations, indicating potential distinct roles in cortical network function development. Moreover, while excitatory and GABAergic dynamics exhibited high correlations, brief deviations from perfect timing were observed. This correlation patterns changed significantly during development and across regions, with the two populations gradually becoming more correlated as they matured. Manipulating inhibition in vivo disrupted these fluctuations, impacting both local activity and the wider functional network.These findings provide valuable insights into the developmental trajectories of spontaneous activity patterns in excitatory and GABAergic cell populations during early postnatal development. The interplay between both neuronal populations plays a critical role in shaping activity patterns, and understanding the underlying mechanisms of their development can provide valuable insights into neurodevelopmental disorders

Personalized data analytics for internet-of-things-based health monitoring

The Internet-of-Things (IoT) has great potential to fundamentally alter the delivery of modern healthcare, enabling healthcare solutions outside the limits of conventional clinical settings. It can offer ubiquitous monitoring to at-risk population groups and allow diagnostic care, preventive care, and early intervention in everyday life. These services can have profound impacts on many aspects of health and well-being. However, this field is still at an infancy stage, and the use of IoT-based systems in real-world healthcare applications introduces new challenges. Healthcare applications necessitate satisfactory quality attributes such as reliability and accuracy due to their mission-critical nature, while at the same time, IoT-based systems mostly operate over constrained shared sensing, communication, and computing resources. There is a need to investigate this synergy between the IoT technologies and healthcare applications from a user-centered perspective. Such a study should examine the role and requirements of IoT-based systems in real-world health monitoring applications. Moreover, conventional computing architecture and data analytic approaches introduced for IoT systems are insufficient when used to target health and well-being purposes, as they are unable to overcome the limitations of IoT systems while fulfilling the needs of healthcare applications. This thesis aims to address these issues by proposing an intelligent use of data and computing resources in IoT-based systems, which can lead to a high-level performance and satisfy the stringent requirements. For this purpose, this thesis first delves into the state-of-the-art IoT-enabled healthcare systems proposed for in-home and in-hospital monitoring. The findings are analyzed and categorized into different domains from a user-centered perspective. The selection of home-based applications is focused on the monitoring of the elderly who require more remote care and support compared to other groups of people. In contrast, the hospital-based applications include the role of existing IoT in patient monitoring and hospital management systems. Then, the objectives and requirements of each domain are investigated and discussed. This thesis proposes personalized data analytic approaches to fulfill the requirements and meet the objectives of IoT-based healthcare systems. In this regard, a new computing architecture is introduced, using computing resources in different layers of IoT to provide a high level of availability and accuracy for healthcare services. This architecture allows the hierarchical partitioning of machine learning algorithms in these systems and enables an adaptive system behavior with respect to the user's condition. In addition, personalized data fusion and modeling techniques are presented, exploiting multivariate and longitudinal data in IoT systems to improve the quality attributes of healthcare applications. First, a real-time missing data resilient decision-making technique is proposed for health monitoring systems. The technique tailors various data resources in IoT systems to accurately estimate health decisions despite missing data in the monitoring. Second, a personalized model is presented, enabling variations and event detection in long-term monitoring systems. The model evaluates the sleep quality of users according to their own historical data. Finally, the performance of the computing architecture and the techniques are evaluated in this thesis using two case studies. The first case study consists of real-time arrhythmia detection in electrocardiography signals collected from patients suffering from cardiovascular diseases. The second case study is continuous maternal health monitoring during pregnancy and postpartum. It includes a real human subject trial carried out with twenty pregnant women for seven months