Extensions and Applications of Ensemble-of-trees Methods in Machine Learning

Ensemble-of-trees algorithms have emerged to the forefront of machine learning due to their ability to generate high forecasting accuracy for a wide array of regression and classification problems. Classic ensemble methodologies such as random forests (RF) and stochastic gradient boosting (SGB) rely on algorithmic procedures to generate fits to data. In contrast, more recent ensemble techniques such as Bayesian Additive Regression Trees (BART) and Dynamic Trees (DT) focus on an underlying Bayesian probability model to generate the fits. These new probability model-based approaches show much promise versus their algorithmic counterparts, but also offer substantial room for improvement. The first part of this thesis focuses on methodological advances for ensemble-of-trees techniques with an emphasis on the more recent Bayesian approaches. In particular, we focus on extensions of BART in four distinct ways. First, we develop a more robust implementation of BART for both research and application. We then develop a principled approach to variable selection for BART as well as the ability to naturally incorporate prior information on important covariates into the algorithm. Next, we propose a method for handling missing data that relies on the recursive structure of decision trees and does not require imputation. Last, we relax the assumption of homoskedasticity in the BART model to allow for parametric modeling of heteroskedasticity. The second part of this thesis returns to the classic algorithmic approaches in the context of classification problems with asymmetric costs of forecasting errors. First we consider the performance of RF and SGB more broadly and demonstrate its superiority to logistic regression for applications in criminology with asymmetric costs. Next, we use RF to forecast unplanned hospital readmissions upon patient discharge with asymmetric costs taken into account. Finally, we explore the construction of stable decision trees for forecasts of violence during probation hearings in court systems

A review of dynamic Bayesian network techniques with applications in healthcare risk modelling

Coping with an ageing population is a major concern for healthcare organisations around the world. The average cost of hospital care is higher than social care for older and terminally ill patients. Moreover, the average cost of social care increases with the age of the patient. Therefore, it is important to make efficient and fair capacity planning which also incorporates patient centred outcomes. Predictive models can provide predictions which their accuracy can be understood and quantified. Predictive modelling can help patients and carers to get the appropriate support services, and allow clinical decision-makers to improve care quality and reduce the cost of inappropriate hospital and Accident and Emergency admissions. The aim of this study is to provide a review of modelling techniques and frameworks for predictive risk modelling of patients in hospital, based on routinely collected data such as the Hospital Episode Statistics database. A number of sub-problems can be considered such as Length-of-Stay and End-of-Life predictive modelling. The methodologies in the literature are mainly focused on addressing the problems using regression methods and Markov models, and the majority lack generalisability. In some cases, the robustness, accuracy and re-usability of predictive risk models have been shown to be improved using Machine Learning methods. Dynamic Bayesian Network techniques can represent complex correlations models and include small probabilities into the solution. The main focus of this study is to provide a review of major time-varying Dynamic Bayesian Network techniques with applications in healthcare predictive risk modelling

A New Scalable, Portable, and Memory-Efficient Predictive Analytics Framework for Predicting Time-to-Event Outcomes in Healthcare

Time-to-event outcomes are prevalent in medical research. To handle these outcomes, as well as censored observations, statistical and survival regression methods are widely used based on the assumptions of linear association; however, clinicopathological features often exhibit nonlinear correlations. Machine learning (ML) algorithms have been recently adapted to effectively handle nonlinear correlations. One drawback of ML models is that they can model idiosyncratic features of a training dataset. Due to this overlearning, ML models perform well on the training data but are not so striking on test data. The features that we choose indirectly influence the performance of ML prediction models. With the expansion of big data in biomedical informatics, appropriate feature engineering and feature selection are vital to ML success. Also, an ensemble learning algorithm helps decrease bias and variance by combining the predictions of multiple models. In this study, we newly constructed a scalable, portable, and memory-efficient predictive analytics framework, fitting four components (feature engineering, survival analysis, feature selection, and ensemble learning) together. Our framework first employs feature engineering techniques, such as binarization, discretization, transformation, and normalization on raw dataset. The normalized feature set was applied to the Cox survival regression that produces highly correlated features relevant to the outcome.The resultant feature set was deployed to “eXtreme gradient boosting ensemble learning” (XGBoost) and Recursive Feature Elimination algorithms. XGBoost uses a gradient boosting decision tree algorithm in which new models are created sequentially that predict the residuals of prior models, which are then added together to make the final prediction. In our experiments, we analyzed a cohort of cardiac surgery patients drawn from a multi-hospital academic health system. The model evaluated 72 perioperative variables that impact an event of readmission within 30 days of discharge, derived 48 significant features, and demonstrated optimum predictive ability with feature sets ranging from 16 to 24. The area under the receiver operating characteristics observed for the feature set of 16 were 0.8816, and 0.9307 at the 35th, and 151st iteration respectively. Our model showed improved performance compared to state-of-the-art models and could be more useful for decision support in clinical settings

Interpretable Machine Learning Model for Clinical Decision Making

Despite machine learning models being increasingly used in medical decision-making and meeting classification predictive accuracy standards, they remain untrusted black-boxes due to decision-makers\u27 lack of insight into their complex logic. Therefore, it is necessary to develop interpretable machine learning models that will engender trust in the knowledge they generate and contribute to clinical decision-makers intention to adopt them in the field. The goal of this dissertation was to systematically investigate the applicability of interpretable model-agnostic methods to explain predictions of black-box machine learning models for medical decision-making. As proof of concept, this study addressed the problem of predicting the risk of emergency readmissions within 30 days of being discharged for heart failure patients. Using a benchmark data set, supervised classification models of differing complexity were trained to perform the prediction task. More specifically, Logistic Regression (LR), Random Forests (RF), Decision Trees (DT), and Gradient Boosting Machines (GBM) models were constructed using the Healthcare Cost and Utilization Project (HCUP) Nationwide Readmissions Database (NRD). The precision, recall, area under the ROC curve for each model were used to measure predictive accuracy. Local Interpretable Model-Agnostic Explanations (LIME) was used to generate explanations from the underlying trained models. LIME explanations were empirically evaluated using explanation stability and local fit (R2). The results demonstrated that local explanations generated by LIME created better estimates for Decision Trees (DT) classifiers

DeepCare: A Deep Dynamic Memory Model for Predictive Medicine

Personalized predictive medicine necessitates the modeling of patient illness and care processes, which inherently have long-term temporal dependencies. Healthcare observations, recorded in electronic medical records, are episodic and irregular in time. We introduce DeepCare, an end-to-end deep dynamic neural network that reads medical records, stores previous illness history, infers current illness states and predicts future medical outcomes. At the data level, DeepCare represents care episodes as vectors in space, models patient health state trajectories through explicit memory of historical records. Built on Long Short-Term Memory (LSTM), DeepCare introduces time parameterizations to handle irregular timed events by moderating the forgetting and consolidation of memory cells. DeepCare also incorporates medical interventions that change the course of illness and shape future medical risk. Moving up to the health state level, historical and present health states are then aggregated through multiscale temporal pooling, before passing through a neural network that estimates future outcomes. We demonstrate the efficacy of DeepCare for disease progression modeling, intervention recommendation, and future risk prediction. On two important cohorts with heavy social and economic burden -- diabetes and mental health -- the results show improved modeling and risk prediction accuracy.Comment: Accepted at JBI under the new name: "Predicting healthcare trajectories from medical records: A deep learning approach

Use of deep learning to develop continuous-risk models for adverse event prediction from electronic health records

Early prediction of patient outcomes is important for targeting preventive care. This protocol describes a practical workflow for developing deep-learning risk models that can predict various clinical and operational outcomes from structured electronic health record (EHR) data. The protocol comprises five main stages: formal problem definition, data pre-processing, architecture selection, calibration and uncertainty, and generalizability evaluation. We have applied the workflow to four endpoints (acute kidney injury, mortality, length of stay and 30-day hospital readmission). The workflow can enable continuous (e.g., triggered every 6 h) and static (e.g., triggered at 24 h after admission) predictions. We also provide an open-source codebase that illustrates some key principles in EHR modeling. This protocol can be used by interdisciplinary teams with programming and clinical expertise to build deep-learning prediction models with alternate data sources and prediction tasks

Characterization of Postoperative Recovery After Cardiac Surgery- Insights into Predicting Individualized Recovery Pattern

Understanding the patterns of postoperative recovery after cardiac surgery is important from several perspectives: to facilitate patient-centered treatment decision making, to inform health care policy targeted to improve postoperative recovery, and to guide patient care after cardiac surgery. Our works aimed to address the following: 1) to summarize existing approaches to measuring and reporting postoperative recovery after cardiac surgery, 2) to develop a framework to efficiently measure patient-reported outcome measures to understand longitudinal recovery process, and 3) to explore ways to summarize the longitudinal recovery data in an actionable way, and 4) to evaluate whether addition of patient information generated through different phases of care would improve the ability to predict patient’s outcome. We first conducted a systematic review of the studies reporting on postoperative recovery after cardiac surgery using patient-reported outcome measures. Our systematic review demonstrated that the current approaches to measuring and reporting recovery as a treatment outcome varied widely across studies. This made synthesis of collective knowledge challenging and highlighted key gaps in knowledge, which we sought to address in our prospective cohort study. We conducted a prospective single-center cohort study of patients after cardiac surgery to measure their recovery trajectory across multiple domains of recovery. Using a digital platform, we measured patient recovery in various domains over 30 days after surgery to visualize a granular evolution of patient recovery after cardiac surgery. We used a latent class analysis to facilitate identification of dominant trajectory patterns that had been obscured in a conventional way of reporting such time-series data using group-level means. For the pain domain, we identified 4 trajectory classes, one of which was a group of patients with persistently high pain trajectory that only became distinguishable from less concerning group after 10 days. Therefore, we obtained a potentially actionable insights to tailoring individualized follow-up timing after surgery to improve the pain control. The prospective study embodied several important features to successfully conducting such studies of patient-reported outcomes. This included the use of digital platform to facilitate efficient data collection extending after hospital discharge, iteratively improving the protocol to optimize patient engagement including evaluation of potential barriers to survey completion, and using latent class analysis to identify dominant patterns of recovery trajectories. We outlined these insights in the protocol manuscript to inform subsequent studies aiming to leverage such a digital platform to measure longitudinal patient-centered outcome. Finally, we evaluated the potential value of incorporating health care data generated in the different phases of patient care in improving the prediction of postoperative outcomes after cardiac surgery. The current standard of risk prediction in cardiac surgery is the Society of Thoracic Surgeons’ (STS) risk model, which only uses patient information available preoperatively. We demonstrated through prediction models fitted on the national STS risk model for coronary artery bypass graft surgery that the addition of intraoperative variables to the conventional preoperative variable set improved the performance of prediction models substantially. Using machine learning approach to such a high-dimensional dataset proved to be marginally important. This work demonstrated the potential value and importance of being able to leverage health care data to continuously update the prediction to inform patient outcomes and guide clinical care. Our work collectively advanced knowledge in several key aspects of postoperative recovery. First, we highlighted the knowledge gap in the existing literature through characterizing the variability in the ways such studies had been conducted. Second, we designed and described a framework to measure postoperative recovery and an analytical approach to informatively characterize longitudinal patient recovery. Third, we employed these designs in a prospective cohort study to measure and analyze recovery trajectories and described clinical insights obtained from the study. Finally, we demonstrated the potential value of a dynamic risk model to iteratively improve its predictive performance by incorporating new data generated as the patient progresses through the phase of care. Such a platform has the potential to individualize patient’s post-acute care in a data-driven manner