Digital phenotyping through multimodal, unobtrusive sensing

The growing adoption of multimodal wearable and mobile devices, such as smartphones and wrist-worn watches has generated an increase in the collection of physiological and behavioural data at scale. This digital phenotyping data enables researchers to make inferences regarding users’ physical and mental health at scale, for the first time. However, translating this data into actionable insights requires computational approaches that turn unlabelled, multimodal time-series sensor data into validated measures that can be interpreted at scale. This thesis describes the derivation of novel computational methods that leverage digital phenotyping data from wearable devices in large-scale populations to infer physical behaviours. These methods combine insights from signal processing, data mining and machine learning alongside domain knowledge in physical activity and sleep epidemiology. First, the inference of sleeping windows in free-living conditions through a heart rate sensing approach is explored. This algorithm is particularly valuable in the absence of ground truth or sleep diaries given its simplicity, adaptability and capacity for personalization. I then explore multistage sleep classification through combined movement and cardiac wearable sensing and machine learning. Further, I demonstrate that postural changes detected through wrist accelerometers can inform habitual behaviours and are valuable complements to traditional, intensity-based physical activity metrics. I then leverage the concomitant responses of heart rate to physical activity that can be captured through multimodal wearable sensors through a self-supervised training task. The resulting embeddings from this task are shown to be useful for the downstream classification of demographic factors, BMI, energy expenditure and cardiorespiratory fitness. Finally, I describe a deep learning model for the adaptive inference of cardiorespiratory fitness (VO2max) using wearable data in free living conditions. I demonstrate the robustness of the model in a large UK population and show the models’ adaptability by evaluating its performance in a subset of the population with repeated measures ~6 years after the original recordings. Together, this work increases the potential of multimodal wearable and mobile sensors for physical activity and behavioural inferences in population studies. In particular, this thesis showcases the potential of using wearable devices to make valuable physical activity, sleep and fitness inferences in large cohort studies. Given the nature of the data collected and the fact that most of this data is currently generated by commercial providers and not research institutes, laying the foundations for responsible data governance and ethical use of these technologies will be critical to building trust and enabling the development of the field of digital phenotyping.I was funded by GlaxoSmithKline and the Engineering and Physical Sciences Research Council. I was also supported by the Alan Turing Institute through their Enrichment Scheme

High-throughput field phenotyping in cereals and implications in plant ecophysiology

[eng] Global climate change effects on agroecosystems together with increasing world population is already threatening food security and endangering ecosystem stability. Meet global food demand with crops production under climate change scenario is the core challenge in plant research nowadays. Thus, there is an urgent need to better understand the underpinning mechanisms of plant acclimation to stress conditions contributing to obtain resilient crops. Also, it is essential to develop new methods in plant research that permit to better characterize non-destructively plant traits of interest. In this sense, the advance in plant phenotyping research by high throughput systems is key to overcome these challenges, while its verification in the field may clear doubts on its feasibility. To this aim, this thesis focused on wheat and secondarily on maize as study species as they make up the major staple crops worldwide. A large panoply of phenotyping methods was employed in these works, ranging from RGB and hyperspectral sensing to metabolomic characterization, besides of other more conventional traits. All research was performed with trials grown in the field and diverse stressor conditions representative of major constrains for plant growth and production were studied: water stress, nitrogen deficiency and disease stress. Our results demonstrated the great potential of leave-to-canopy color traits captured by RGB sensors for in-field phenotyping, as they were accurate and robust indicators of grain yield in wheat and maize under disease and nitrogen deficiency conditions and of leaf nitrogen concentration in maize. On the other hand, the characterization of the metabolome of wheat tissues contributed to elucidate the metabolic mechanisms triggered by water stress and their relationship with high yielding performance, providing some potential biomarkers for higher yields and stress adaptation. Spectroscopic studies in wheat highlighted that leaf dorsoventrality may affect more than water stress on the reflected spectrum and consequently the performance of the multispectral/hyperspectral approaches to assess yield or any other relevant phenotypic trait. Anatomy, pigments and water changes were responsible of reflectance differences and the existence of leaf-side-specific responses were discussed. Finally, the use of spectroscopy for the estimation of the metabolite profiles of wheat organs showed promising for many metabolites which could pave the way for a new generation phenotyping. We concluded that future phenotyping may benefit from these findings in both the low-cost and straightforward methods and the more complex and frontier technologies.[cat] Els efectes del canvi climàtic sobre els agro-ecosistemes i l’increment de la població mundial posa en risc la seguretat alimentària i l’estabilitat dels ecosistemes. Actualment, satisfer les demandes de producció d’aliments sota l’escenari del canvi climàtic és el repte central a la Biologia Vegetal. Per això, és indispensable entendre els mecanismes subjacents de l’aclimatació a l’estrès que permeten obtenir cultius resilients. També és precís desenvolupar nou mètodes de recerca que permetin caracteritzar de manera no destructiva els trets d’interès. L’avenç del fenotipat vegetal amb sistemes d’alt rendiment és clau per abordar aquests reptes. La present tesi s’enfoca en el blat i secundàriament en el panís com a espècies d’estudi ja que constitueixen els cultius bàsics arreu del món. Un ampli ventall de mètodes de fenotipat s’han utilitzat, des sensors RGB a híper-espectrals fins a la caracterització metabolòmica. La recerca s’ha dut a terme en assajos de camp i s’han avaluat diversos tipus d’estrès representatius de les majors limitacions pel creixement i producció vegetal: estrès hídric i biòtic i deficiència de nitrogen. Els resultats demostraren el gran potencial dels trets del color RGB (des de la planta a la capçada) pel fenotipat de camp, ja que foren indicadors precisos del rendiment a blat i panís sota condicions de malaltia i deficiència de nitrogen i de la concentració de nitrogen foliar a panís. La caracterització metabolòmica de teixits de blat contribuí a esbrinar els processos metabòlics endegats per l’estrès hídric i la seva relació amb comportament genotípic, proporcionant bio-marcadors potencials per rendiments més alts i l’adaptació a l’estrès. Estudis espectroscòpics en blat van demostrar que la dorsoventralitat pot afectar més que l’estrès hídric sobre l’espectre de reflectància i consegüentment sobre el comportament de les aproximacions multi/híper-espectrals per avaluar el rendiment i d’altres trets fenotípics com anatòmics i contingut de pigments. Finalment, l’ús de l’espectroscòpia per l’estimació del contingut metabòlic als teixits de blat resulta prometedor per molts metabòlits, la qual cosa obre les portes per a un fenotipat de nova generació. El fenotipat pot beneficiar-se d’aquestes troballes, tant en els mètodes de baix cost com de les tecnologies més sofisticades i d’avantguarda

Generating Reliable and Responsive Observational Evidence: Reducing Pre-analysis Bias

A growing body of evidence generated from observational data has demonstrated the potential to influence decision-making and improve patient outcomes. For observational evidence to be actionable, however, it must be generated reliably and in a timely manner. Large distributed observational data networks enable research on diverse patient populations at scale and develop new sound methods to improve reproducibility and robustness of real-world evidence. Nevertheless, the problems of generalizability, portability and scalability persist and compound. As analytical methods only partially address bias, reliable observational research (especially in networks) must address the bias at the design stage (i.e., pre-analysis bias) including the strategies for identifying patients of interest and defining comparators. This thesis synthesizes and enumerates a set of challenges to addressing pre-analysis bias in observational studies and presents mixed-methods approaches and informatics solutions for overcoming a number of those obstacles. We develop frameworks, methods and tools for scalable and reliable phenotyping including data source granularity estimation, comprehensive concept set selection, index date specification, and structured data-based patient review for phenotype evaluation. We cover the research on potential bias in the unexposed comparator definition including systematic background rates estimation and interpretation, and definition and evaluation of the unexposed comparator. We propose that the use of standardized approaches and methods as described in this thesis not only improves reliability but also increases responsiveness of observational evidence. To test this hypothesis, we designed and piloted a Data Consult Service - a service that generates new on-demand evidence at the bedside. We demonstrate that it is feasible to generate reliable evidence to address clinicians’ information needs in a robust and timely fashion and provide our analysis of the current limitations and future steps needed to scale such a service

Optimization of AI models as the Main Component in Prospective Edge Intelligence Applications

Artificial Intelligence (AI) is a successful paradigm with application in many fields; however, there can be some challenging scenarios like the deployment of AI models in remote locations or with limited connectivity, possibly needing to perform inference closer to where data is collected. A potential solution is the study of ways to optimize AI models, for deployment of intelligent algorithms closer to the edge. This thesis focuses on applications of AI that need to have characteristics that make them suitable for implementation on portable devices (e.g., aeroponics container, drone, mobile robot); thus, the development and implementation of custom models, and their optimization (i.e., reduction in size and inference time) is of upmost importance and the main goal of this dissertation. For this task, a number of options have been explored, including developing techniques to select relevant features from the samples that the model analyzes, and pruning and quantization. Therefore, this thesis proposes a scheme for the development, implementation, and optimization of custom AI models used mainly in agriculture, so that they have the desired characteristics that are needed for their deployment in edge devices. This main goal is fulfilled by implementing a number of sequential steps that include the validation of the hypothesis that there is at least an AI model capable of generating useful predictions for the applications being studied, the exploration and implementation of an approach for their optimization, and their final implementation in hardware of limited resources. The main contributions of this thesis are on the general workflow for optimization of custom models, as well as in the proposed scheme for feature selection based on model interpretability approaches. This carries most of the novelty of the thesis, since we have not found previous implementations of these ideas, at least in the field under study. This optimization is mainly based on a feature selection approach, but finally complemented with pruning and quantization. The implementation of some of these models on an edge-like device, demonstrates the feasibility of this approach. Finally, although this thesis tries to be a self-contained work, encompassing all the aspects required to go from AI model design to implementation on an edge device, there are some aspects that could be further studied, analyzed, and improved. Furthermore, it is almost impossible to keep the pace with all the new developments in the fields of AI, edge computing, hardware and software tools, etc. which opens the field for new discussions and proposals. This work tries to fill some gaps and to propose some ideas that hopefully will be useful for future researchers in the development of new technologies and solutions

Applying computer vision for detection of diseases in plants

Early detection and quantification of diseases in food plants are critical to agriculture industry and national food security. However, limitation in technology and cost has limited the success of applying Computer Vision in Plant Science. This research builds on the recent advance of Machine Learning, GPU and smartphones to tackle the problem of fast and low cost diagnosis of plant disease. In particular, we choose soybean as the subject for applying automatic disease detection. The reason is because soybean is an important crop for the state of Iowa and an important source of food for America. The plant is however, highly vulnerable to several type of diseases. This thesis consists of two sub-analyses of soybean diseases, which are: First, detection of a single disease in soybean, particularly Sudden Death Syndrome (SDS) with high detail (including location and severity). Second, detection of multiple diseases in soybean, using mobile phones which are resource- constraine

Secure and robust machine learning for healthcare: A survey

Recent years have witnessed widespread adoption of machine learning (ML)/deep learning (DL) techniques due to their superior performance for a variety of healthcare applications ranging from the prediction of cardiac arrest from one-dimensional heart signals to computer-aided diagnosis (CADx) using multi-dimensional medical images. Notwithstanding the impressive performance of ML/DL, there are still lingering doubts regarding the robustness of ML/DL in healthcare settings (which is traditionally considered quite challenging due to the myriad security and privacy issues involved), especially in light of recent results that have shown that ML/DL are vulnerable to adversarial attacks. In this paper, we present an overview of various application areas in healthcare that leverage such techniques from security and privacy point of view and present associated challenges. In addition, we present potential methods to ensure secure and privacy-preserving ML for healthcare applications. Finally, we provide insight into the current research challenges and promising directions for future research