Stochastic Dynamic Optimization Under Ambiguity

Stochastic dynamic optimization methods are powerful mathematical tools for informing sequential decision-making in environments where the outcomes of decisions are uncertain. For instance, the Markov decision process (MDP) has found success in many application areas, including the evaluation and design of treatment and screening protocols for medical decision making. However, the usefulness of these models is only as good as the data used to parameterize them, and multiple competing data sources are common in many application areas. Unfortunately, the recommendations that result from the optimization process can be sensitive to the data used and thus, susceptible to the impacts of ambiguity in the choices regarding the model's construction. To address the issue of ambiguity in MDPs, we introduce the Multi-model MDP (MMDP) which generalizes a standard MDP by allowing for multiple models of the rewards and transition probabilities. The solution of the MMDP is a policy that considers the performance with respect to the different models and allows for the decision-maker (DM) to explicitly trade-off conflicting sources of data. In this thesis, we study this problem in three parts. In the first part, we study the weighted value problem (WVP) in which the DM’s objective is to find a single policy that maximizes the weighted value of expected rewards in each model. We identify two important variants of this problem: the non-adaptive WVP in which the DM must specify the decision-making strategy before the outcome of ambiguity is observed and the adaptive WVP in which the DM is allowed to adapt to the outcomes of ambiguity. To solve these problems, we develop exact methods and fast approximation methods supported by error bounds. Finally, we illustrate the effectiveness and the scalability of our approach using a case study in preventative blood pressure and cholesterol management that accounts for conflicting published cardiovascular risk models. In the second part, we leverage the special structure of the non-adaptive WVP to design exact decomposition methods for solving MMDPs with a larger number of models. We present a branch-and-cut approach to solve a mixed-integer programming formulation of the problem and a custom branch-and-bound approach. Numerical experiments show that a customized implementation of branch-and-bound significantly outperforms branch-and-cut and allows for the solution of MMDPs with larger numbers of models. In the third part, we extend the MMDP beyond the WVP to consider other objective functions that are sensitive to the ambiguity arising from the existence of multiple models. We modify the branch-and-bound procedure to solve these alternate formulations and compare the resulting policies to policies found using tractable heuristics. Using two case studies, we show that the solution to the mean value problem, wherein all parameters take on their mean values, can perform quite well with respect to several measures of performance under ambiguity. In summary, in this dissertation, we present new methods for stochastic dynamic optimization under ambiguity. We represent ambiguity through multiple plausible models of the MDP. We analyze alternative forms of the problems, develop solution methods, and identify properties of the optimal solutions that provide insight into the effects of ambiguity on optimal policies. Although we illustrate our methods on decision-making for medical treatment and machine maintenance, the methods we present in this thesis can be applied to other domains in which optimal sequential decision-making uncertainty is clouded by ambiguity.PHDIndustrial & Operations EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/149947/1/steimle_1.pd

The Cardiac TBX5 Interactome Reveals a Chromatin Remodeling Network Essential for Cardiac Septation

Human mutations in the cardiac transcription factor gene TBX5 cause Congenital Heart Disease (CHD), however the underlying mechanism is unknown. We report characterization of the endogenous TBX5 cardiac interactome and demonstrate that TBX5, long considered a transcriptional activator, interacts biochemically and genetically with the Nucleosome Remodeling and Deacetylase (NuRD) repressor complex. Incompatible gene programs are repressed by TBX5 in the developing heart. CHD missense mutations that disrupt the TBX5-NuRD interaction cause depression of a subset of repressed genes. Furthermore, the TBX5-NuRD interaction is required for heart development. Phylogenetic analysis showed that the TBX5-NuRD interaction domain evolved during early diversification of vertebrates, simultaneous with the evolution of cardiac septation. Collectively, this work defines a TBX5-NuRD interaction essential to cardiac development and the evolution of the mammalian heart, and when altered may contribute to human CHD

The Implications of State Aggregation on the Utility of Estimated Markov Decision Process Models

Markov Decision Processes (MDPs) are mathematical models of sequential decision-making under uncertainty that have found applications in healthcare, manufacturing, logistics, and others. For many applications, the natural state space for the system is too large to be used in the model effectively, so modelers will aggregate this natural state space into a smaller superstate space to ensure computational tractability and improve confidence in the accuracy of the estimated transition probability matrix. That modeler’s choice of state space leads to a trade-off between accuracy and precision in transition probability estimates; while coarser discretization of the state space gives more observations in each state to estimate the transition probability matrix, this comes that the cost of precision in the state characterization and resulting policy recommendations. In this paper, we show that there is highest potential for policy improvement on larger state spaces, but that aggregated MDPs are preferable under limited data

Machine Learning Basics with Applications to Email Spam Detection

Mentor: Arye Nehorai From the Washington University Undergraduate Research Digest: WUURD, Volume 9, Issue 2, Spring 2014. Published by the Office of Undergraduate Research, Joy Zalis Kiefer Director of Undergraduate Research and Assistant Dean in the College of Arts & Sciences; Kristin Sobotka, Editor

Demand Response Management Using Game Theory for the Smart Grid

Mentor: Arye Nehorai From the Washington University Undergraduate Research Digest: WUURD, Volume 8, Issue 2, Spring 2013. Published by the Office of Undergraduate Research, Joy Zalis Kiefer Director of Undergraduate Research and Assistant Dean in the College of Arts & Sciences; Kristin Sobotka, Editor

Students’ Preferences for Returning to Colleges and Universities During the COVID-19 Pandemic: A Discrete Choice Experiment

Importance: Due to the COVID-19 pandemic, institutions of higher education (IHEs) are weighing decisions about when and how to reopen their campuses for the Spring 2021 term. Schools are revisiting their plans to use in-person, online, or hybrid (a mixture of in-person and online) modes of course delivery, as well as their safety plans. However, there is still limited knowledge about how to properly plan for campus reopening decisions, including course delivery and campus safety, to maintain enrollment and keep students and faculty safe. Objectives: To assess 1) students’ willingness to comply with health protocols and contrast to their perception of their classmates’ compliance, 2) whether students preferred in-person or online learning during a pandemic, and 3) The importance weights of different aspects of campus operations (i.e., modes of course delivery and safety plans) for students when they decide to enroll or defer. Design, setting, and participants: An internet-based survey of college students took place from June 25, 2020 to July 10, 2020. Participants included 398 industrial engineering students at a medium-size public university in Atlanta, Georgia. The survey included a discrete choice experiment with questions that asked students to choose whether to enroll or defer when presented with hypothetical scenarios related to Fall 2020 modes of course delivery and aspects of campus safety. The survey also asked students about expected compliance with health protocols, whether they preferred in-person or online courses, and sociodemographic information. Main outcomes and measures: We estimated students’ willingness to comply with potential health protocols, choices between in-person and online learning, and the importance of different modes of course delivery and safety measures when deciding to enroll or defer. Results: The response rate of students who participated in the survey was 20.8%. A latent class model showed three classes of students: those who were “low-concern” (comprising a 29% expected share of the sample), those who were “moderate-concern” (54%) and those who were “high-concern” (17%). We found that scenarios that offered an on-campus experience with large classes delivered online and small classes delivered in-person, strict safety protocols in terms of mask-wearing, testing, and residence halls, and lenient safety protocols in terms of social gatherings were broadly the scenarios with the highest expected enrollment probabilities. The decision to enroll or defer for all students was largely determined by the mode of delivery for courses and the safety measures on campus around COVID-19 testing and mask-wearing. A logistic regression model showed that higher perceived risk of infection of COVID-19, better living suitability for online courses, being older, and less risk seeking were significant factors for a person to choose online learning. Students stated for themselves and their classmates that they would comply with some but not all health protocols against COVID-19, especially those limiting social gatherings. Conclusions and relevance: The majority of students indicated a preference to enroll during the COVID-19 pandemic so long as sufficient safety measures are put in place and all classes were not entirely in-person. As IHEs consider different options for campus operations during pandemics, they should consider the heterogenous preferences among their students. Offering flexibility in course modes may be a way to appeal to many students who vary in terms of their concern about the COVID-19 pandemic. At the same time, since students overall preferred some safety measures placed around mask-wearing and COVID-19 testing on campus, IHEs may want to recommend or require wearing masks and doing some surveillance tests for all students, faculty, and staff. Students were expecting themselves and their fellow classmates to comply with some but not all health protocols, which may help IHEs identify protocols that need more education and awareness, like the limits on social gatherings and the practice of social distancing at social gatherings

Students' preferences for returning to colleges and universities during the COVID-19 pandemic: A discrete choice experiment.

ImportanceWhen an emerging infectious disease outbreak occurs, such as COVID-19, institutions of higher education (IHEs) must weigh decisions about how to operate their campuses. These decisions entail whether campuses should remain open, how courses should be delivered (in-person, online, or a mixture of the two), and what safety plans should be enacted for those on campus. These issues have weighed heavily on campus administrators during the on-going COVID-19 pandemic. However, there is still limited knowledge about how such decisions affect students' enrollment decisions and campus safety in practice when considering compliance.ObjectivesTo assess 1) students' willingness to comply with health protocols and contrast their perception of their classmates' compliance, 2) whether students prefer in-person or online learning during a pandemic, and 3) the importance weights of different aspects of campus operations (i.e., modes of course delivery and safety plans) for students when they decide to enroll or defer.Design setting and participantsAn internet-based survey of college students took place from June 25, 2020 to July 10, 2020. Participants included 398 industrial engineering students at the Georgia Institute of Technology, a medium-size public university in Atlanta, Georgia. The survey included a discrete choice experiment with questions that asked students to choose whether to enroll or defer when presented with hypothetical scenarios related to Fall 2020 modes of course delivery and aspects of campus safety. The survey also asked students about expected compliance with health protocols, whether they preferred in-person or online courses, and sociodemographic information.Main outcomes and measuresWe examine students' willingness to comply with potential health protocols. We estimated logistic regression models to infer significant factors that lead to a student's choice between in-person and online learning. Additionally, we estimated discrete choice models to infer the importance of different modes of course delivery and safety measures to students when deciding to enroll or defer.ResultsThe survey response rate was 20.8%. A latent class model showed three classes of students: those who were "low-concern" (comprising a 29% expected share of the sample), those who were "moderate-concern" (54%) and those who were "high-concern" (17%). We found that scenarios that offered an on-campus experience with large classes delivered online and small classes delivered in-person, strict safety protocols in terms of mask-wearing, testing, and residence halls, and lenient safety protocols in terms of social gatherings were broadly the scenarios with the highest expected enrollment probabilities. The decision to enroll or defer for all students was largely determined by the mode of delivery for courses and the safety measures on campus around COVID-19 testing and mask-wearing. A logistic regression model showed that a higher perceived risk of infection of COVID-19, a more suitable home environment, being older, and being less risk-seeking were significant factors for a person to choose online learning. Students stated for themselves and their classmates that they would comply with some but not all health protocols against COVID-19, especially those limiting social gatherings.Conclusions and relevanceThe majority of students indicated a preference to enroll during the COVID-19 pandemic so long as sufficient safety measures were put in place and all classes were not entirely in-person. As IHEs consider different options for campus operations during pandemics, they should consider the heterogeneous preferences among their students. Offering flexibility in course modes may be a way to appeal to many students who vary in terms of their concern about the pandemic. At the same time, since students overall preferred some safety measures placed around mask-wearing and COVID-19 testing on campus, IHEs may want to recommend or require wearing masks and doing some surveillance tests for all students, faculty, and staff. Students were expecting themselves and their fellow classmates to comply with some but not all health protocols, which may help IHEs identify protocols that need more education and awareness, like limits on social gatherings and the practice of social distancing at social gatherings

Revisiting the small-world property of co-enrollment networks: A network analysis of hybrid course delivery strategies

Infectious disease outbreaks bring new challenges to campus operations at institutions of higher education (IHEs). At the beginning of the COVID-19 pandemic, as a precaution against COVID-19 transmission on their campuses, many IHEs shifted to entirely remote courses in the Spring 2020 semester. As these institutions prepared for the Fall 2020 semester, their administrations evaluated the extent to which remote courses should continue. While remote courses may reduce the number of students who interact on campus and thus mitigate the risk of disease spread, there may be academic and financial downsides. Methodology: With de-identified enrollment, course schedule, and student profile data from the Fall 2019 semester at Georgia Institute of Technology (Georgia Tech), we construct a co-enrollment network, which is a representation of student connections through courses. In this co-enrollment network, we observe the small-world property in which the average path length among reachable student pairs is short and nearly all students are connected to each other in the main component of the network through multiple independent paths. We also see high connectivity between different majors. Then, we apply various hybrid instructional mode strategies to this co-enrollment network and use network connectivity and in-person instruction metrics to compare these strategies from both health and academic perspectives, by their impacts on various groups of students, and based on the trade-off between network connectivity and in-person instruction. Results: Although these strategies result in decreased network connectivity, they do not remove the small-world property. When comparing the strategies in terms of network connectivity, we find three different strategy types to be the most favorable. Even under these strategies, the percentage of students with in-person courses that could be exposed rises quickly with initial infection rates. When comparing the strategies in terms of in-person instruction, the most favorable strategies select a fixed percentage of courses for each subject to be delivered remotely. We then consider the impacts of these strategies on various groups of students to evaluate if there may be disparate impacts across these groups. In general, we find that strategies with more remote courses result in less disparate impacts on network connectivity and in-person instruction across these groups. In addition, we generally see less in-person instruction for lower-level students (i.e. first-year students and sophomores) compared to upper-level students. Regarding the trade-off between network connectivity and in-person instruction, the only strategy that is suboptimal under all scenarios in this study is the 1000s and 4000s only strategy, which keeps all 1000- and 4000-level courses in-person. This is because of its large quantity of remote courses, the large 1000- and 4000-level courses, the highly central 1000-level courses, and the high connectivity among first-year students and seniors. Importance: By quantitatively comparing network connectivity and in-person instruction for these various strategies, campus administrators can weigh the physical health, academic performance, and mental health of students, along with the financial well-being of their academic institution when responding to infectious disease outbreaks

Benefit and harm of intensive blood pressure treatment: Derivation and validation of risk models using data from the SPRINT and ACCORD trials

Background: Intensive blood pressure (BP) treatment can avert cardiovascular disease (CVD) events but can cause some serious adverse events. We sought to develop and validate risk models for predicting absolute risk difference (increased risk or decreased risk) for CVD events and serious adverse events from intensive BP therapy. A secondary aim was to test if the statistical method of elastic net regularization would improve the estimation of risk models for predicting absolute risk difference, as compared to a traditional backwards variable selection approach. Methods and findings Cox models were derived from SPRINT trial data and validated on ACCORD-BP trial data to estimate risk of CVD events and serious adverse events; the models included terms for intensive BP treatment and heterogeneous response to intensive treatment. The Cox models were then used to estimate the absolute reduction in probability of CVD events (benefit) and absolute increase in probability of serious adverse events (harm) for each individual from intensive treatment. We compared the method of elastic net regularization, which uses repeated internal cross-validation to select variables and estimate coefficients in the presence of collinearity, to a traditional backwards variable selection approach. Data from 9,069 SPRINT participants with complete data on covariates were utilized for model development, and data from 4,498 ACCORD-BP participants with complete data were utilized for model validation. Participants were exposed to intensive (goal systolic pressure < 120 mm Hg) versus standard (<140 mm Hg) treatment. Two composite primary outcome measures were evaluated: (i) CVD events/deaths (myocardial infarction, acute coronary syndrome, stroke, congestive heart failure, or CVD death), and (ii) serious adverse events (hypotension, syncope, electrolyte abnormalities, bradycardia, or acute kidney injury/failure). The model for CVD chosen through elastic net regularization included interaction terms suggesting that older age, black race, higher diastolic BP, and higher lipids were associated with greater CVD risk reduction benefits from intensive treatment, while current smoking was associated with fewer benefits. The model for serious adverse events chosen through elastic net regularization suggested that male sex, current smoking, statin use, elevated creatinine, and higher lipids were associated with greater risk of serious adverse events from intensive treatment. SPRINT participants in the highest predicted benefit subgroup had a number needed to treat (NNT) of 24 to prevent 1 CVD event/death over 5 years (absolute risk reduction [ARR] = 0.042, 95% CI: 0.018, 0.066; P = 0.001), those in the middle predicted benefit subgroup had a NNT of 76 (ARR = 0.013, 95% CI: −0.0001, 0.026; P = 0.053), and those in the lowest subgroup had no significant risk reduction (ARR = 0.006, 95% CI: −0.007, 0.018; P = 0.71). Those in the highest predicted harm subgroup had a number needed to harm (NNH) of 27 to induce 1 serious adverse event (absolute risk increase [ARI] = 0.038, 95% CI: 0.014, 0.061; P = 0.002), those in the middle predicted harm subgroup had a NNH of 41 (ARI = 0.025, 95% CI: 0.012, 0.038; P < 0.001), and those in the lowest subgroup had no significant risk increase (ARI = −0.007, 95% CI: −0.043, 0.030; P = 0.72). In ACCORD-BP, participants in the highest subgroup of predicted benefit had significant absolute CVD risk reduction, but the overall ACCORD-BP participant sample was skewed towards participants with less predicted benefit and more predicted risk than in SPRINT. The models chosen through traditional backwards selection had similar ability to identify absolute risk difference for CVD as the elastic net models, but poorer ability to correctly identify absolute risk difference for serious adverse events. A key limitation of the analysis is the limited sample size of the ACCORD-BP trial, which expanded confidence intervals for ARI among persons with type 2 diabetes. Additionally, it is not possible to mechanistically explain the physiological relationships explaining the heterogeneous treatment effects captured by the models, since the study was an observational secondary data analysis. Conclusions: We found that predictive models could help identify subgroups of participants in both SPRINT and ACCORD-BP who had lower versus higher ARRs in CVD events/deaths with intensive BP treatment, and participants who had lower versus higher ARIs in serious adverse events