Deep Offline Reinforcement Learning for Real-World Treatment Optimization Applications

There is increasing interest in data-driven approaches for dynamically choosing optimal treatment strategies in many chronic disease management and critical care applications. Reinforcement learning methods are well-suited to this sequential decision-making problem, but must be trained and evaluated exclusively on retrospective medical record datasets as direct online exploration is unsafe and infeasible. Despite this requirement, the vast majority of dynamic treatment optimization studies use off-policy RL methods (e.g., Double Deep Q Networks (DDQN) or its variants) that are known to perform poorly in purely offline settings. Recent advances in offline RL, such as Conservative Q-Learning (CQL), offer a suitable alternative. But there remain challenges in adapting these approaches to real-world applications where suboptimal examples dominate the retrospective dataset and strict safety constraints need to be satisfied. In this work, we introduce a practical transition sampling approach to address action imbalance during offline RL training, and an intuitive heuristic to enforce hard constraints during policy execution. We provide theoretical analyses to show that our proposed approach would improve over CQL. We perform extensive experiments on two real-world tasks for diabetes and sepsis treatment optimization to compare performance of the proposed approach against prominent off-policy and offline RL baselines (DDQN and CQL). Across a range of principled and clinically relevant metrics, we show that our proposed approach enables substantial improvements in expected health outcomes and in consistency with relevant practice and safety guidelines

Baseline characteristics of participants in the Pre-Diabetes Interventions and Continued Tracking to Ease-out Diabetes (Pre-DICTED) Program

OBJECTIVE: The Pre-Diabetes Interventions and Continued Tracking to Ease-out Diabetes (Pre-DICTED) Program is a diabetes prevention trial comparing the diabetes conversion rate at 3 years between the intervention group, which receives the incentivized lifestyle intervention program with stepwise addition of metformin, and the control group, which receives the standard of care. We describe the baseline characteristics and compare Pre-DICTED participants with other diabetes prevention trials cohort. RESEARCH DESIGN AND METHODS: Participants were aged between 21 and 64 years, overweight (body mass index (BMI) ≥23.0 kg/m2), and had pre-diabetes (impaired fasting glucose (IFG) and/or impaired glucose tolerance (IGT)). RESULTS: A total of 751 participants (53.1% women) were randomized. At baseline, mean (SD) age was 52.5 (8.5) years and mean BMI (SD) was 29.0 (4.6) kg/m2. Twenty-three per cent had both IFG and IGT, 63.9% had isolated IGT, and 13.3% had isolated IFG. Ethnic Asian Indian participants were more likely to report a family history of diabetes and had a higher waist circumference, compared with Chinese and Malay participants. Women were less likely than men to meet the physical activity recommendations (≥150 min of moderate-intensity physical activity per week), and dietary intake varied with both sex and ethnicity. Compared with other Asian diabetes prevention studies, the Pre-DICTED cohort had a higher mean age and BMI. CONCLUSION: The Pre-DICTED cohort represents subjects at high risk of diabetes conversion. The study will evaluate the effectiveness of a community-based incentivized lifestyle intervention program in an urban Asian context.Peer reviewe

Design and baseline characteristics of the finerenone in reducing cardiovascular mortality and morbidity in diabetic kidney disease trial

Background: Among people with diabetes, those with kidney disease have exceptionally high rates of cardiovascular (CV) morbidity and mortality and progression of their underlying kidney disease. Finerenone is a novel, nonsteroidal, selective mineralocorticoid receptor antagonist that has shown to reduce albuminuria in type 2 diabetes (T2D) patients with chronic kidney disease (CKD) while revealing only a low risk of hyperkalemia. However, the effect of finerenone on CV and renal outcomes has not yet been investigated in long-term trials. Patients and Methods: The Finerenone in Reducing CV Mortality and Morbidity in Diabetic Kidney Disease (FIGARO-DKD) trial aims to assess the efficacy and safety of finerenone compared to placebo at reducing clinically important CV and renal outcomes in T2D patients with CKD. FIGARO-DKD is a randomized, double-blind, placebo-controlled, parallel-group, event-driven trial running in 47 countries with an expected duration of approximately 6 years. FIGARO-DKD randomized 7,437 patients with an estimated glomerular filtration rate >= 25 mL/min/1.73 m(2) and albuminuria (urinary albumin-to-creatinine ratio >= 30 to <= 5,000 mg/g). The study has at least 90% power to detect a 20% reduction in the risk of the primary outcome (overall two-sided significance level alpha = 0.05), the composite of time to first occurrence of CV death, nonfatal myocardial infarction, nonfatal stroke, or hospitalization for heart failure. Conclusions: FIGARO-DKD will determine whether an optimally treated cohort of T2D patients with CKD at high risk of CV and renal events will experience cardiorenal benefits with the addition of finerenone to their treatment regimen. Trial Registration: EudraCT number: 2015-000950-39; ClinicalTrials.gov identifier: NCT02545049

Real‐World Systolic and Diastolic Blood Pressure Levels and Cardiovascular Mortality in Patients With Type 2 Diabetes—Results From a Large Registry Cohort in Asia

BACKGROUND Elevated blood pressure (BP) is associated with increased risk of cardiovascular mortality. However, there is ongoing debate whether intensive BP lowering may paradoxically increase the risk of cardiovascular disease (CVD), especially in patients with type 2 diabetes (T2D). We investigated the association of BP with risk of CVD mortality in patients with T2D. METHODS AND RESULTS We used data on 83 721 patients with T2D from a multi‐institutional diabetes registry in Singapore from 2013 to 2019. BP was analyzed as categories and restricted cubic splines using Cox multivariable regression analysis stratified by preexisting CVD and age (120 to 129 mm Hg. Diastolic BP levels >90 mm Hg were significantly associated with CVD mortality in those aged ≥65 years. In addition, diastolic BP <70 mm Hg was associated with a significantly higher risk of CVD mortality in all patients. CONCLUSIONS In patients with T2D, clinic systolic BP levels ≥130 mm Hg or diastolic BP levels ≥90 mm Hg are associated with higher risk of CVD mortality. Diastolic BP <70 mm Hg is also associated with the risk of adverse CVD outcomes, although reverse causality cannot be ruled out

Longitudinal HbA1c trajectory modelling reveals the association of HbA1c and risk of hospitalization for heart failure for patients with type 2 diabetes mellitus.

BackgroundInconsistent conclusions in past studies on the association between poor glycaemic control and the risk of hospitalization for heart failure (HHF) have been reported largely due to the analysis of non-trajectory-based HbA1c values. Trajectory analysis can incorporate the effects of HbA1c variability across time, which may better elucidate its association with macrovascular complications. Furthermore, studies analysing the relationship between HbA1c trajectories from diabetes diagnosis and the occurrence of HHF are scarce.MethodsThis is a prospective cohort study of the SingHealth Diabetes Registry (SDR). 17,389 patients diagnosed with type 2 diabetes mellitus (T2DM) from 2013 to 2016 with clinical records extending to the end of 2019 were included in the latent class growth analysis to extract longitudinal HbA1c trajectories. Association between HbA1c trajectories and risk of first known HHF is quantified with the Cox Proportional Hazards (PH) model.Results5 distinct HbA1c trajectories were identified as 1. low stable (36.1%), 2. elevated stable (40.4%), 3. high decreasing (3.5%), 4. high with a sharp decline (10.8%), and 5. moderate decreasing (9.2%) over the study period of 7 years. Poorly controlled HbA1c trajectories (Classes 3, 4, and 5) are associated with a higher risk of HHF. Using the diabetes diagnosis time instead of a commonly used pre-defined study start time or time from recruitment has an impact on HbA1c clustering results.ConclusionsFindings suggest that tracking the evolution of HbA1c with time has its importance in assessing the HHF risk of T2DM patients, and T2DM diagnosis time as a baseline is strongly recommended in HbA1c trajectory modelling. To the authors' knowledge, this is the first study to identify an association between HbA1c trajectories and HHF occurrence from diabetes diagnosis time

A study to evaluate the prevalence of impaired awareness of hypoglycaemia in adults with type 2 diabetes in outpatient clinic in a tertiary care centre in Singapore

Background: Impaired awareness of hypoglycaemia (IAH) predisposes affected patients to severe hypoglycaemia. There are few data on prevalence of IAH in adults with insulin-treated type 2 diabetes in Asia. We aim to ascertain the prevalence of IAH among insulin-treated patients with type 2 diabetes in an outpatient clinic in a tertiary care centre in Singapore. Methods: A total of 374 patients with insulin-treated type 2 diabetes attending the outpatient diabetes clinic in a tertiary referral centre in Singapore were recruited over a 4-month period. Participants completed a questionnaire to document baseline characteristics and assess their hypoglycaemia awareness status, using a combination of the Clarke, Gold and Pedersen-Bjergaard methods. Results: Using the Clarke, Gold and Pedersen-Bjergaard methods, prevalence of IAH in our cohort was 9.6%, 13.4% and 33.2% respectively. Overall, 7.2% of participants suffered from severe hypoglycaemia in the preceding year. The IAH group had more episodes of severe hypoglycaemia across all three methods, compared with the normal awareness group ( p < 0.01). There were no significant differences in mean HbA1c, duration of diabetes and insulin treatment between the IAH and normal awareness groups. Conclusions: IAH is prevalent in adults with insulin-treated type 2 diabetes in Asia, and is associated with significantly increased risk of severe hypoglycaemia

Audit of Diabetes-Dependent Quality of Life (ADDQoL) &lsqb;Chinese Version for Singapore&rsqb; Questionnaire: Reliability and Validity among Singaporeans with Type 2 Diabetes Mellitus

Background: The Audit of Diabetes-Dependent Quality of Life (ADDQoL) questionnaire is an individualized instrument that measures the impact of diabetes mellitus on quality of life (QOL). With the worldwide increase in the number of Chinese people diagnosed with diabetes, we anticipated that a Chinese-language version of the ADDQoL would be urgently needed. Objective: To evaluate the reliability and validity of the ADDQoL (Chinese version for Singapore) among Chinese-speaking Singaporeans with type 2 diabetes mellitus (T2DM). Methods: Chinese versions of the ADDQoL, EuroQoL-Visual Analogue Scale (EQ-VAS), EQ-5D and SF-6D were administered to Chinese-speaking participants with T2DM (aged ≥21 years) at a tertiary acute-care hospital by convenience sampling. The ADDQoL was assessed for the following: internal consistency (Cronbach's alpha); test-retest reliability (intraclass correlation coefficient &lsqb;ICC&rsqb;); factor structure; known-groups validity (insulin requiring vs non-insulin requiring, with vs without diabetes-related complications, overweight/obese vs not overweight/obese); and convergent and divergent validity (with EQ-VAS, EQ-5D and SF-6D). The usefulness of weighting and 'not applicable' (NA) options (key features of ADDQoL) were also evaluated. Results: In 88 participants (58% male, mean &lsqb;SD&rsqb; age 56.6 &lsqb;11.74&rsqb; years), the mean (SD) ADDQoL average weighted impact (AWI) score was -2.613 (1.899). Cronbach's alpha was 0.941 and the ICC was 0.955 (95% CI 0.812, 0.990). In confirmatory factor analysis, the hypothesized one-factor solution was supported. ADDQoL AWI scores correlated strongly with ADDQoL diabetes-dependent global QOL scores (Spearman's rank correlation coefficient &lsqb;rs&rsqb; - 0.5983) and weakly with generic measures (rs - -0.028 for ADDQoL present global QOL scores, 0.310 for EQ-VAS, 0.164 for EQ-5D and 0.281 for SF-6D). Participants who required insulin, those with diabetes-related complications and those who were overweight/obese reported lower AWI scores, but the differences were not statistically significant. Importance scores of zero were assigned 1-28% of the time and the NA options were selected 3-49% of the time. Conclusions: The ADDQoL is reliable and probably valid for assessing QOL among Chinese-speaking Singaporeans with T2DM, although known-groups validity warrants further investigation.Quality-of-life-rating-scales, Type-2-diabetes-mellitus

High incidence of undetected low sensor glucose events among elderly patients with type 2 diabetes more than a decade on after the ACCORD study.

ObjectiveHypoglycaemia leads to significant morbidity and impacts negatively on quality of life, especially in elderly people with increased frailty.The aims of this study were to determine the prevalence of low interstitial fluid glucose (IFG) in patients with tightly controlled type 2 diabetes (T2D), and to evaluate whether there were differences in burden of low IFG between sulphonylurea and insulin treated groups.MethodsA Freestyle Libre-Pro sensor was used for sampling of the IFG continuously. Patients were blinded to the IFG levels. The sensor was returned to the investigators after a 2-week period and the data were downloaded for analysis.ResultsThere was a total of 69 patients (median age 72 years (IQR 69-74)) - 40 were sulfonylurea-treated and 29 insulin-treated. 781 low sensor glucose events (ConclusionsThis study demonstrates that elderly patients with tightly-controlled T2D have a significant number of low sensor glucose events which go by undetected

Progressive endoplasmic reticulum stress over time due to human insulin gene mutation contributes to pancreatic beta cell dysfunction

10.1007/s00125-021-05530-3DIABETOLOGIA64112534-254