Expert-augmented machine learning.

Machine learning is proving invaluable across disciplines. However, its success is often limited by the quality and quantity of available data, while its adoption is limited by the level of trust afforded by given models. Human vs. machine performance is commonly compared empirically to decide whether a certain task should be performed by a computer or an expert. In reality, the optimal learning strategy may involve combining the complementary strengths of humans and machines. Here, we present expert-augmented machine learning (EAML), an automated method that guides the extraction of expert knowledge and its integration into machine-learned models. We used a large dataset of intensive-care patient data to derive 126 decision rules that predict hospital mortality. Using an online platform, we asked 15 clinicians to assess the relative risk of the subpopulation defined by each rule compared to the total sample. We compared the clinician-assessed risk to the empirical risk and found that, while clinicians agreed with the data in most cases, there were notable exceptions where they overestimated or underestimated the true risk. Studying the rules with greatest disagreement, we identified problems with the training data, including one miscoded variable and one hidden confounder. Filtering the rules based on the extent of disagreement between clinician-assessed risk and empirical risk, we improved performance on out-of-sample data and were able to train with less data. EAML provides a platform for automated creation of problem-specific priors, which help build robust and dependable machine-learning models in critical applications

Using data-driven rules to predict mortality in severe community acquired pneumonia

Prediction of patient-centered outcomes in hospitals is useful for performance benchmarking, resource allocation, and guidance regarding active treatment and withdrawal of care. Yet, their use by clinicians is limited by the complexity of available tools and amount of data required. We propose to use Disjunctive Normal Forms as a novel approach to predict hospital and 90-day mortality from instance-based patient data, comprising demographic, genetic, and physiologic information in a large cohort of patients admitted with severe community acquired pneumonia. We develop two algorithms to efficiently learn Disjunctive Normal Forms, which yield easy-to-interpret rules that explicitly map data to the outcome of interest. Disjunctive Normal Forms achieve higher prediction performance quality compared to a set of state-of-the-art machine learning models, and unveils insights unavailable with standard methods. Disjunctive Normal Forms constitute an intuitive set of prediction rules that could be easily implemented to predict outcomes and guide criteria-based clinical decision making and clinical trial execution, and thus of greater practical usefulness than currently available prediction tools. The Java implementation of the tool JavaDNF will be publicly available. © 2014 Wu et al

The Use of EHR data in Early Detection Systems: A Case in Sepsis and In-Hospital Mortality Prediction

In this thesis, we aim to use electronic health records (EHRs) to predict sepsis and in-hospital mortality by using machine learning algorithms. We first explored EHRs dataset and performed data cleansing. Then, we extracted 57 features using data of vital signs and white blood cell (WBC) count. Two classification algorithms (i.e., random forest and neural network) were used to develop predictive models using the data from the first few hours after admission to predict sepsis and in-hospital mortality. In addition, we also used the data collected in the last few hours before sepsis developed to predict sepsis.The results show promise in early prediction of sepsis and possibly providing an opportunity for directing early intervention efforts to prevent or treat sepsis

Machine-Learning Model for Mortality Prediction in Patients With Community-Acquired Pneumonia Development and Validation Study

Background: Artificial intelligence tools and techniques such as machine learning (ML) are increasingly seen as a suitable manner in which to increase the prediction capacity of currently available clinical tools, including prognostic scores. However, studies evaluating the efficacy of ML methods in enhancing the predictive capacity of existing scores for community-acquired pneumonia (CAP) are limited. We aimed to apply and validate a causal probabilistic network (CPN) model to predict mortality in patients with CAP. Research question: Is a CPN model able to predict mortality in patients with CAP better than the commonly used severity scores? Study design and methods: This was a derivation-validation retrospective study conducted in two Spanish university hospitals. The ability of a CPN designed to predict mortality in sepsis (SepsisFinder [SeF]), and adapted for CAP (SeF-ML), to predict 30-day mortality was assessed and compared with other scoring systems (Pneumonia Severity Index [PSI], Sequential Organ Failure Assessment [SOFA], quick Sequential Organ Failure Assessment [qSOFA], and CURB-65 criteria [confusion, urea, respiratory rate, BP, age ≥ 65 years]). The SeF models are proprietary software. Differences between receiver operating characteristic curves were assessed by the DeLong method for correlated receiver operating characteristic curves. Results: The derivation cohort comprised 4,531 patients, and the validation cohort consisted of 1,034 patients. In the derivation cohort, the areas under the curve (AUCs) of SeF-ML, CURB-65, SOFA, PSI, and qSOFA were 0.801, 0.759, 0.671, 0.799, and 0.642, respectively, for 30-day mortality prediction. In the validation study, the AUC of SeF-ML was 0.826, concordant with the AUC (0.801) in the derivation data (P = .51). The AUC of SeF-ML was significantly higher than those of CURB-65 (0.764; P = .03) and qSOFA (0.729, P = .005). However, it did not differ significantly from those of PSI (0.830; P = .92) and SOFA (0.771; P = .14). Interpretation: SeF-ML shows potential for improving mortality prediction among patients with CAP, using structured health data. Additional external validation studies should be conducted to support generalizability