Optimizing cancer screening with POMDPs

Current clinical decision-making relies heavily both upon the experience of a physician and the recommendations of evidence-based practice guidelines, the latter often informed by population-level policies. Yet with the heightened complexity of patient care given newer types of data and longitudinal observations (e.g., from the electronic health record, EHR), as well as the goal of more individually-tailored healthcare, medical decision-making is increasingly complicated. This issue is particularly true in cancer with emergent techniques for early detection and personalized treatment. This research establishes an informatics-based framework to inform optimal cancer screening through sequential decision-making methods. This dissertation develops tools to formulate a partially observable Markov decision process (POMDP) model, enabling each component to be learned from a dataset: dynamic Bayesian networks (DBNs) are embedded in the POMDP learning process to estimate transition and observations probabilities; inverse reinforcement learning is used to learn a reward function from experts’ prior decisions, and risk prediction models are employed to compute individualized initial beliefs about disease state. The result is a comprehensive approach to implementing sequential decision making agents. These methods are validated using large datasets from lung and breast cancer screening efforts, demonstrating the potential to help tailor and improve early cancer prediction while reducing false positive tests

Optimizing cancer screening with POMDPs

Current clinical decision-making relies heavily both upon the experience of a physician and the recommendations of evidence-based practice guidelines, the latter often informed by population-level policies. Yet with the heightened complexity of patient care given newer types of data and longitudinal observations (e.g., from the electronic health record, EHR), as well as the goal of more individually-tailored healthcare, medical decision-making is increasingly complicated. This issue is particularly true in cancer with emergent techniques for early detection and personalized treatment. This research establishes an informatics-based framework to inform optimal cancer screening through sequential decision-making methods. This dissertation develops tools to formulate a partially observable Markov decision process (POMDP) model, enabling each component to be learned from a dataset: dynamic Bayesian networks (DBNs) are embedded in the POMDP learning process to estimate transition and observations probabilities; inverse reinforcement learning is used to learn a reward function from experts’ prior decisions, and risk prediction models are employed to compute individualized initial beliefs about disease state. The result is a comprehensive approach to implementing sequential decision making agents. These methods are validated using large datasets from lung and breast cancer screening efforts, demonstrating the potential to help tailor and improve early cancer prediction while reducing false positive tests

Evaluating the Impact of Uncertainty on Risk Prediction: Towards More Robust Prediction Models.

Risk prediction models are crucial for assessing the pretest probability of cancer and are applied to stratify patient management strategies. These models are frequently based on multivariate regression analysis, requiring that all risk factors be specified, and do not convey the confidence in their predictions. We present a framework for uncertainty analysis that accounts for variability in input values. Uncertain or missing values are replaced with a range of plausible values. These ranges are used to compute individualized risk confidence intervals. We demonstrate our approach using the Gail model to evaluate the impact of uncertainty on management decisions. Up to 13% of cases (uncertain) had a risk interval that falls within the decision threshold (e.g., 1.67% 5-year absolute risk). A small number of cases changed from low- to high-risk when missing values were present. Our analysis underscores the need for better communication of input assumptions that influence the resulting predictions

Prediction of lung cancer incidence on the low-dose computed tomography arm of the National Lung Screening Trial: A dynamic Bayesian network

IntroductionIdentifying high-risk lung cancer individuals at an early disease stage is the most effective way of improving survival. The landmark National Lung Screening Trial (NLST) demonstrated the utility of low-dose computed tomography (LDCT) imaging to reduce mortality (relative to x-ray screening). As a result of the NLST and other studies, imaging-based lung cancer screening programs are now being implemented. However, LDCT interpretation results in a high number of false positives. A set of dynamic Bayesian networks (DBN) were designed and evaluated to provide insight into how longitudinal data can be used to help inform lung cancer screening decisions.MethodsThe LDCT arm of the NLST dataset was used to build and explore five DBNs for high-risk individuals. Three of these DBNs were built using a backward construction process, and two using structure learning methods. All models employ demographic, smoking status, cancer history, family lung cancer history, exposure risk factors, comorbidities related to lung cancer, and LDCT screening outcome information. Given the uncertainty arising from lung cancer screening, a cancer state-space model based on lung cancer staging was utilized to characterize the cancer status of an individual over time. The models were evaluated on balanced training and test sets of cancer and non-cancer cases to deal with data imbalance and overfitting.ResultsResults were comparable to expert decisions. The average area under the curve (AUC) of the receiver operating characteristic (ROC) for the three intervention points of the NLST trial was higher than 0.75 for all models. Evaluation of the models on the complete LDCT arm of the NLST dataset (N = 25, 486) demonstrated satisfactory generalization. Consensus of predictions over similar cases is reported in concordance statistics between the models’ and the physicians’ predictions. The models’ predictive ability with respect to missing data was also evaluated with the sample of cases that missed the second screening exam of the trial (N = 417). The DBNs outperformed comparison models such as logistic regression and naïve Bayes.ConclusionThe lung cancer screening DBNs demonstrated high discrimination and predictive power with the majority of cancer and non-cancer cases

Generating Reward Functions Using IRL Towards Individualized Cancer Screening

Cancer screening can benefit from individualized decision-making tools that decrease overdiagnosis. The heterogeneity of cancer screening participants advocates the need for more personalized methods. Partially observable Markov decision processes (POMDPs), when defined with an appropriate reward function, can be used to suggest optimal, individualized screening policies. However, determining an appropriate reward function can be challenging. Here, we propose the use of inverse reinforcement learning (IRL) to form rewards functions for lung and breast cancer screening POMDPs. Using experts (physicians) retrospective screening decisions for lung and breast cancer screening, we developed two POMDP models with corresponding reward functions. Specifically, the maximum entropy (MaxEnt) IRL algorithm with an adaptive step size was employed to learn rewards more efficiently; and combined with a multiplicative model to learn state-action pair rewards for a POMDP. The POMDP screening models were evaluated based on their ability to recommend appropriate screening decisions before the diagnosis of cancer. The reward functions learned with the MaxEnt IRL algorithm, when combined with POMDP models in lung and breast cancer screening, demonstrate performance comparable to experts. The Cohen’s Kappa score of agreement between the POMDPs and physicians’ predictions was high in breast cancer and had a decreasing trend in lung cancer

Data-driven prediction of continuous renal replacement therapy survival

Abstract Continuous renal replacement therapy (CRRT) is a form of dialysis prescribed to severely ill patients who cannot tolerate regular hemodialysis. However, as the patients are typically very ill to begin with, there is always uncertainty whether they will survive during or after CRRT treatment. Because of outcome uncertainty, a large percentage of patients treated with CRRT do not survive, utilizing scarce resources and raising false hope in patients and their families. To address these issues, we present a machine learning-based algorithm to predict short-term survival in patients being initiated on CRRT. We use information extracted from electronic health records from patients who were placed on CRRT at multiple institutions to train a model that predicts CRRT survival outcome; on a held-out test set, the model achieves an area under the receiver operating curve of 0.848 (CI = 0.822–0.870). Feature importance, error, and subgroup analyses provide insight into bias and relevant features for model prediction. Overall, we demonstrate the potential for predictive machine learning models to assist clinicians in alleviating the uncertainty of CRRT patient survival outcomes, with opportunities for future improvement through further data collection and advanced modeling