Deep learning based short-term total cloud cover forecasting.

In this research, we conduct deep learning based Total Cloud Cover (TCC) forecasting using satellite images. The proposed system employs the Otsu's method for cloud segmentation and Long Short-Term Memory (LSTM) variant models for TCC prediction. Specifically, a region-based Otsu's method is used to segment clouds from satellite images. A time-series dataset is generated using the TCC information extracted from each image in image sequences using a new feature extraction method. The generated time series data are subsequently used to train several LSTM variant models, i.e. LSTM, bi-directional LSTM and Convolutional Neural Network (CNN)-LSTM, for future TCC forecasting. Our approach achieves impressive average RMSE scores with multi-step forecasting, i.e. 0.0543 and 0.0823, with respect to both the first half of daytime and full daytime TCC forecasting on a given day, using the generated dataset

Landmark Tracking in Liver US images Using Cascade Convolutional Neural Networks with Long Short-Term Memory

This study proposed a deep learning-based tracking method for ultrasound (US) image-guided radiation therapy. The proposed cascade deep learning model is composed of an attention network, a mask region-based convolutional neural network (mask R-CNN), and a long short-term memory (LSTM) network. The attention network learns a mapping from a US image to a suspected area of landmark motion in order to reduce the search region. The mask R-CNN then produces multiple region-of-interest (ROI) proposals in the reduced region and identifies the proposed landmark via three network heads: bounding box regression, proposal classification, and landmark segmentation. The LSTM network models the temporal relationship among the successive image frames for bounding box regression and proposal classification. To consolidate the final proposal, a selection method is designed according to the similarities between sequential frames. The proposed method was tested on the liver US tracking datasets used in the Medical Image Computing and Computer Assisted Interventions (MICCAI) 2015 challenges, where the landmarks were annotated by three experienced observers to obtain their mean positions. Five-fold cross-validation on the 24 given US sequences with ground truths shows that the mean tracking error for all landmarks is 0.65+/-0.56 mm, and the errors of all landmarks are within 2 mm. We further tested the proposed model on 69 landmarks from the testing dataset that has a similar image pattern to the training pattern, resulting in a mean tracking error of 0.94+/-0.83 mm. Our experimental results have demonstrated the feasibility and accuracy of our proposed method in tracking liver anatomic landmarks using US images, providing a potential solution for real-time liver tracking for active motion management during radiation therapy

A meta-learning algorithm for respiratory flow prediction from FBG-based wearables in unrestrained conditions

The continuous monitoring of an individual's breathing can be an instrument for the assessment and enhancement of human wellness. Specific respiratory features are unique markers of the deterioration of a health condition, the onset of a disease, fatigue and stressful circumstances. The early and reliable prediction of high-risk situations can result in the implementation of appropriate intervention strategies that might be lifesaving. Hence, smart wearables for the monitoring of continuous breathing have recently been attracting the interest of many researchers and companies. However, most of the existing approaches do not provide comprehensive respiratory information. For this reason, a meta-learning algorithm based on LSTM neural networks for inferring the respiratory flow from a wearable system embedding FBG sensors and inertial units is herein proposed. Different conventional machine learning approaches were implemented as well to ultimately compare the results. The meta-learning algorithm turned out to be the most accurate in predicting respiratory flow when new subjects are considered. Furthermore, the LSTM model memory capability has been proven to be advantageous for capturing relevant aspects of the breathing pattern. The algorithms were tested under different conditions, both static and dynamic, and with more unobtrusive device configurations. The meta-learning results demonstrated that a short one-time calibration may provide subject-specific models which predict the respiratory flow with high accuracy, even when the number of sensors is reduced. Flow RMS errors on the test set ranged from 22.03 L/min, when the minimum number of sensors was considered, to 9.97 L/min for the complete setting (target flow range: 69.231 ± 21.477 L/min). The correlation coefficient r between the target and the predicted flow changed accordingly, being higher (r = 0.9) for the most comprehensive and heterogeneous wearable device configuration. Similar results were achieved even with simpler settings which included the thoracic sensors (r ranging from 0.84 to 0.88; test flow RMSE = 10.99 L/min, when exclusively using the thoracic FBGs). The further estimation of respiratory parameters, i.e., rate and volume, with low errors across different breathing behaviors and postures proved the potential of such approach. These findings lay the foundation for the implementation of reliable custom solutions and more sophisticated artificial intelligence-based algorithms for daily life health-related applications

Machine learning to model health with multimodal mobile sensor data

The widespread adoption of smartphones and wearables has led to the accumulation of rich datasets, which could aid the understanding of behavior and health in unprecedented detail. At the same time, machine learning and specifically deep learning have reached impressive performance in a variety of prediction tasks, but their use on time-series data appears challenging. Existing models struggle to learn from this unique type of data due to noise, sparsity, long-tailed distributions of behaviors, lack of labels, and multimodality. This dissertation addresses these challenges by developing new models that leverage multi-task learning for accurate forecasting, multimodal fusion for improved population subtyping, and self-supervision for learning generalized representations. We apply our proposed methods to challenging real-world tasks of predicting mental health and cardio-respiratory fitness through sensor data. First, we study the relationship of passive data as collected from smartphones (movement and background audio) to momentary mood levels. Our new training pipeline, which combines different sensor data into a low-dimensional embedding and clusters longitudinal user trajectories as outcome, outperforms traditional approaches based solely on psychology questionnaires. Second, motivated by mood instability as a predictor of poor mental health, we propose encoder-decoder models for time-series forecasting which exploit the bi-modality of mood with multi-task learning. Next, motivated by the success of general-purpose models in vision and language tasks, we propose a self-supervised neural network ready-to-use as a feature extractor for wearable data. To this end, we set the heart rate responses as the supervisory signal for activity data, leveraging their underlying physiological relationship and show that the resulting task-agnostic embeddings can generalize in predicting structurally different downstream outcomes through transfer learning (e.g. BMI, age, energy expenditure), outperforming unsupervised autoencoders and biomarkers. Finally, acknowledging fitness as a strong predictor of overall health, which, however, can only be measured with expensive instruments (e.g., a VO2max test), we develop models that enable accurate prediction of fine-grained fitness levels with wearables in the present, and more importantly, its direction and magnitude almost a decade later. All proposed methods are evaluated on large longitudinal datasets with tens of thousands of participants in the wild. The models developed and the insights drawn in this dissertation provide evidence for a better understanding of high-dimensional behavioral and physiological data with implications for large-scale health and lifestyle monitoring.The Department of Computer Science and Technology at the University of Cambridge through the EPSRC through Grant DTP (EP/N509620/1), and the Embiricos Trust Scholarship of Jesus College Cambridg

Digital phenotyping through multimodal, unobtrusive sensing

The growing adoption of multimodal wearable and mobile devices, such as smartphones and wrist-worn watches has generated an increase in the collection of physiological and behavioural data at scale. This digital phenotyping data enables researchers to make inferences regarding users’ physical and mental health at scale, for the first time. However, translating this data into actionable insights requires computational approaches that turn unlabelled, multimodal time-series sensor data into validated measures that can be interpreted at scale. This thesis describes the derivation of novel computational methods that leverage digital phenotyping data from wearable devices in large-scale populations to infer physical behaviours. These methods combine insights from signal processing, data mining and machine learning alongside domain knowledge in physical activity and sleep epidemiology. First, the inference of sleeping windows in free-living conditions through a heart rate sensing approach is explored. This algorithm is particularly valuable in the absence of ground truth or sleep diaries given its simplicity, adaptability and capacity for personalization. I then explore multistage sleep classification through combined movement and cardiac wearable sensing and machine learning. Further, I demonstrate that postural changes detected through wrist accelerometers can inform habitual behaviours and are valuable complements to traditional, intensity-based physical activity metrics. I then leverage the concomitant responses of heart rate to physical activity that can be captured through multimodal wearable sensors through a self-supervised training task. The resulting embeddings from this task are shown to be useful for the downstream classification of demographic factors, BMI, energy expenditure and cardiorespiratory fitness. Finally, I describe a deep learning model for the adaptive inference of cardiorespiratory fitness (VO2max) using wearable data in free living conditions. I demonstrate the robustness of the model in a large UK population and show the models’ adaptability by evaluating its performance in a subset of the population with repeated measures ~6 years after the original recordings. Together, this work increases the potential of multimodal wearable and mobile sensors for physical activity and behavioural inferences in population studies. In particular, this thesis showcases the potential of using wearable devices to make valuable physical activity, sleep and fitness inferences in large cohort studies. Given the nature of the data collected and the fact that most of this data is currently generated by commercial providers and not research institutes, laying the foundations for responsible data governance and ethical use of these technologies will be critical to building trust and enabling the development of the field of digital phenotyping.I was funded by GlaxoSmithKline and the Engineering and Physical Sciences Research Council. I was also supported by the Alan Turing Institute through their Enrichment Scheme

Human-centred artificial intelligence for mobile health sensing:challenges and opportunities

Advances in wearable sensing and mobile computing have enabled the collection of health and well-being data outside of traditional laboratory and hospital settings, paving the way for a new era of mobile health. Meanwhile, artificial intelligence (AI) has made significant strides in various domains, demonstrating its potential to revolutionize healthcare. Devices can now diagnose diseases, predict heart irregularities and unlock the full potential of human cognition. However, the application of machine learning (ML) to mobile health sensing poses unique challenges due to noisy sensor measurements, high-dimensional data, sparse and irregular time series, heterogeneity in data, privacy concerns and resource constraints. Despite the recognition of the value of mobile sensing, leveraging these datasets has lagged behind other areas of ML. Furthermore, obtaining quality annotations and ground truth for such data is often expensive or impractical. While recent large-scale longitudinal studies have shown promise in leveraging wearable sensor data for health monitoring and prediction, they also introduce new challenges for data modelling. This paper explores the challenges and opportunities of human-centred AI for mobile health, focusing on key sensing modalities such as audio, location and activity tracking. We discuss the limitations of current approaches and propose potential solutions

Assessing the Impact of Blood Pressure on Cardiac Function Using Interpretable Biomarkers and Variational Autoencoders

Maintaining good cardiac function for as long as possible is a major concern for healthcare systems worldwide and there is much interest in learning more about the impact of different risk factors on cardiac health. The aim of this study is to analyze the impact of systolic blood pressure (SBP) on cardiac function while preserving the interpretability of the model using known clinical biomarkers in a large cohort of the UK Biobank population. We propose a novel framework that combines deep learning based estimation of interpretable clinical biomarkers from cardiac cine MR data with a variational autoencoder (VAE). The VAE architecture integrates a regression loss in the latent space, which enables the progression of cardiac health with SBP to be learnt. Results on 3,600 subjects from the UK Biobank show that the proposed model allows us to gain important insight into the deterioration of cardiac function with increasing SBP, identify key interpretable factors involved in this process, and lastly exploit the model to understand patterns of positive and adverse adaptation of cardiac function

The Temporal and Frequent Pattern Mining Analysis and Machine Learning Forecasting on Mobile Sourced Urban Air Pollutants

Ground-level ozone and atmospheric fine particles (PM2.5) have been recognized as critical air pollutants that act as important contributors to the toxicity of anthropogenic air pollution in urban areas. To limit the adverse impacts on public health and ecosystems of ground-level ozone and PM2.5, it is necessary and imperative to identify a practical and effective way to predict the upcoming pollution concentration levels accurately. Under this need, various research was conducted aiming to perform the forecasting of ground-level ozone and PM2.5 that mainly utilized the time-series and neural network analysis. In the meantime, machine learning is also adopted in analysis and forecasting in existing research, which is, however, associated with some limitations that are not easily overcome. (1) The majority of existing forecasting models are highly dependent on time-series inputs without considering the influencing factors of the air pollutants. While a relatively accurate prediction may be provided, the influencing factors of the air pollution level caused by real-world complexity are neglected. (2) The existing forecasting models are mainly focused on the short-term estimation, while some of them need to use the previous prediction as a part of the input, which increased the system complexity and decreased the computational efficiency and accuracy. (3) The accurate annual hourly air pollution level forecasting ability is seldomly achieved. The objective of this research is to propose a systematical methodology to forecast the long-term hourly future air pollution concentration levels through historical data considering the concentration influencing factors. To achieve this research goal, a series of methodologies to analyze the historical air pollution concentration by temporal characteristics and frequent pattern data mining algorithms are introduced. The association rules of air pollution concentration levels and the influencing factors are revealed. A systematical air pollution level forecasting approach based on supervised machine learning algorithms with the ability to predict the annual hourly value is proposed and evaluated. To quantify and validate the results, a case study was conducted in the Houston region with the collection and analysis of ten years of historical environmental, meteorological, and transportation-related data. From the results of this research, (1) the complex correlations between the influencing factors and air pollution concentration levels are quantified and presented. (2) The association rules between each dependant and independent parameters are calculated. (3) The supervised machine learning algorithm pool is created and evaluated. And (4), an accurate long-term hourly air pollution level machine learning forecasting procedure is proposed. The innovative methodology of this research is advanced in computation complexity with high accuracy when compared with the existing models, which could be easily applied to similar regions for various types of air pollution concentration level forecasting

Effects of meteorology on PM10 concentrations: a comparative assessment of machine learning methods

Administrative decisions regarding the application of measures to address air quality issues have to rely both on present observation and future predictions of the concentration of various pollutants. Since PM10 is one of the most critical pollutants, the ability to provide accurate forecasts for its concentration, when required, is crucial in order to enforce the necessary measures at the right time. Together with the pattern of emission sources which is present in a geographical area, meteorological conditions can significantly affect the concentration of pollutants in air, since they can favour the dispersion or, on the other hand, the build-up of those compounds. It is possible then to predict (at least partially) the concentration of PM10 in air using meteorological variables as predictors. In fact, various statistical models have been proposed for accomplishing similar tasks on a number of geographical regions and urban areas, with varying results. The set of meteorological variables that have been considered in those cases included various predictors, measured both in the day of interest and in the previous ones. Sometimes also some non-meteorological descriptors (e.g. time-related variables) that are grossly related to the variation of the emission patterns have been considered as input variables for those models. In this work an analysis of the relationship between meteorology-related variables and PM10 concentration levels in the capitals of the provinces of Emilia-Romagna has been performed in order to understand how the meteorological conditions affect PM10 concentration. Then the considered meteorological variables have been input as predictors to statistical regression models based on machine learning in order to obtain predictions for the daily mean value of PM10 concentration