Ambiguous Dynamic Treatment Regimes: A Reinforcement Learning Approach

A main research goal in various studies is to use an observational data set and provide a new set of counterfactual guidelines that can yield causal improvements. Dynamic Treatment Regimes (DTRs) are widely studied to formalize this process. However, available methods in finding optimal DTRs often rely on assumptions that are violated in real-world applications (e.g., medical decision-making or public policy), especially when (a) the existence of unobserved confounders cannot be ignored, and (b) the unobserved confounders are time-varying (e.g., affected by previous actions). When such assumptions are violated, one often faces ambiguity regarding the underlying causal model. This ambiguity is inevitable, since the dynamics of unobserved confounders and their causal impact on the observed part of the data cannot be understood from the observed data. Motivated by a case study of finding superior treatment regimes for patients who underwent transplantation in our partner hospital and faced a medical condition known as New Onset Diabetes After Transplantation (NODAT), we extend DTRs to a new class termed Ambiguous Dynamic Treatment Regimes (ADTRs), in which the causal impact of treatment regimes is evaluated based on a "cloud" of causal models. We then connect ADTRs to Ambiguous Partially Observable Mark Decision Processes (APOMDPs) and develop Reinforcement Learning methods, which enable using the observed data to efficiently learn an optimal treatment regime. We establish theoretical results for these learning methods, including (weak) consistency and asymptotic normality. We further evaluate the performance of these learning methods both in our case study and in simulation experiments

Optimal Dynamic Control of Queueing Networks: Emergency Departments, the W Service Network, and Supply Chains under Disruptions.

Many systems in both the service and manufacturing sectors can be modeled and analyzed as queueing networks. In such systems, control and design is often an important issue that may significantly affect the performance. This dissertation focuses on the development of innovative techniques for the design and control of such systems. Special attention is given to real-world applications in (a) the design and control of patient flow in the hospital emergency departments, (b) design and control of service/call centers, and (c) the design and control of supply chains under disruption risks. With respect to application (a), using hospital data, analytical models, and simulation analyses we show how (1) better patient prioritization, (2) enhanced triage systems, and (3) improved patient flow designs allow emergency departments to significantly improve their performance with respect to both operational efficiency and patient safety. Regarding application (b), we give specific attention to a two-server and three-demand class network in the shape of a ``W'' with random server disruption and repair times. Studying this network, we show how effective control and design strategies that efficiently make use of (partial) flexibility of servers can be implemented to achieve high performance and resilience to server disruptions. In addition to establishing stability properties of different known control mechanisms, a new heuristic policy, termed Largest Expected Workload Cost (LEWC), is proposed and its performance is extensively benchmarked with respect to other widely used polices. Regarding application (c), we demonstrate how supply chains can boost their performance using better control and design strategies that efficiently take into account supply disruption risks. Motivated by several real-world examples of disruptions, production flexibility, and supply contracts within supply chains, we model the informational and operational flexibility approaches to designing a resilient supply chain. By analyzing optimal ordering policies, sourcing strategies, and the optimal levels of back-up capacity reservation contracts, various disruption risk mitigation strategies are considered and compared, and new insights into the design of resilient supply chains are provided.PHDIndustrial and Operations EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/94002/1/soroush_1.pd

Reevaluating the Role of Race and Ethnicity in Diabetes Screening

There is active debate over whether to consider patient race and ethnicity when estimating disease risk. By accounting for race and ethnicity, it is possible to improve the accuracy of risk predictions, but there is concern that their use may encourage a racialized view of medicine. In diabetes risk models, despite substantial gains in statistical accuracy from using race and ethnicity, the gains in clinical utility are surprisingly modest. These modest clinical gains stem from two empirical patterns: first, the vast majority of individuals receive the same screening recommendation regardless of whether race or ethnicity are included in risk models; and second, for those who do receive different screening recommendations, the difference in utility between screening and not screening is relatively small. Our results are based on broad statistical principles, and so are likely to generalize to many other risk-based clinical decisions.Comment: 11 pages, 4 figure

Use of imputation and decision modeling to improve diagnosis and management of patients at risk for new-onset diabetes after transplantation

BACKGROUND New-onset diabetes after transplantation (NODAT) is a complication of solid organ transplantation. We sought to determine the extent to which NODAT goes undiagnosed over the course of 1 year following transplantation, analyze missed or later-diagnosed cases of NODAT due to poor hemoglobin A1c (HbA1c) and fasting blood glucose (FBG) collection, and to estimate the impact that improved NODAT screening metrics may have on long-term outcomes. MATERIAL AND METHODS This was a retrospective study utilizing 3 datasets from a single center on kidney, liver, and heart transplantation patients. Retrospective analysis was supplemented with an imputation procedure to account for missing data and project outcomes under perfect information. In addition, the data were used to inform a simulation model used to estimate life expectancy and cost-effectiveness of a hypothetical intervention. RESULTS Estimates of NODAT incidence increased from 27% to 31% in kidney transplantation patients, from 31% to 40% in liver transplantation patients, and from 45% to 67% in heart transplantation patients, when HbA1c and FBG were assumed to be collected perfectly at all points. Perfect screening for kidney transplantation patients was cost-saving, while perfect screening for liver and heart transplantation patients was cost-effective at a willingness-to-pay threshold of $100 000 per life-year. CONCLUSIONS Improved collection of HbA1c and FBG is a cost-effective method for detecting many additional cases of NODAT within the first year alone. Additional research into both improved glucometric monitoring as well as effective strategies for mitigating NODAT risk will become increasingly important to improve health in this population.Accepted manuscrip

Characterization of Remitting and Relapsing Hyperglycemia in Post-Renal-Transplant Recipients.

Hyperglycemia following solid organ transplant is common among patients without pre-existing diabetes mellitus (DM). Post-transplant hyperglycemia can occur once or multiple times, which if continued, causes new-onset diabetes after transplantation (NODAT).To study if the first and recurrent incidence of hyperglycemia are affected differently by immunosuppressive regimens, demographic and medical-related risk factors, and inpatient hyperglycemic conditions (i.e., an emphasis on the time course of post-transplant complications).We conducted a retrospective analysis of 407 patients who underwent kidney transplantation at Mayo Clinic Arizona. Among these, there were 292 patients with no signs of DM prior to transplant. For this category of patients, we evaluated the impact of (1) immunosuppressive drugs (e.g., tacrolimus, sirolimus, and steroid), (2) demographic and medical-related risk factors, and (3) inpatient hyperglycemic conditions on the first and recurrent incidence of hyperglycemia in one year post-transplant. We employed two versions of Cox regression analyses: (1) a time-dependent model to analyze the recurrent cases of hyperglycemia and (2) a time-independent model to analyze the first incidence of hyperglycemia.Age (P = 0.018), HDL cholesterol (P = 0.010), and the average trough level of tacrolimus (P<0.0001) are significant risk factors associated with the first incidence of hyperglycemia, while age (P<0.0001), non-White race (P = 0.002), BMI (P = 0.002), HDL cholesterol (P = 0.003), uric acid (P = 0.012), and using steroid (P = 0.007) are the significant risk factors for the recurrent cases of hyperglycemia.This study draws attention to the importance of analyzing the risk factors associated with a disease (specially a chronic one) with respect to both its first and recurrent incidence, as well as carefully differentiating these two perspectives: a fact that is currently overlooked in the literature

Classification of literature based on diabetogenic effect of immunosuppressive drugs.

<p>Classification of literature based on diabetogenic effect of immunosuppressive drugs.</p

Number of patients who used immunosuppressive drugs at months 1, 4, and 12.

<p>Such patients are further classified as having hyperglycemia (HG) or not at that specific time points.</p

Description of groups formed by risk factors.

<p>Description of groups formed by risk factors.</p

Demographic and baseline characteristics of patients at the time of transplant.

<p>^a mean ± standard deviation,</p><p>^b versus non-white (including Native American, Hispanic, and Black races),</p><p>^c versus cadaveric.</p><p>Demographic and baseline characteristics of patients at the time of transplant.</p