Deep Offline Reinforcement Learning for Real-World Treatment Optimization Applications

There is increasing interest in data-driven approaches for dynamically choosing optimal treatment strategies in many chronic disease management and critical care applications. Reinforcement learning methods are well-suited to this sequential decision-making problem, but must be trained and evaluated exclusively on retrospective medical record datasets as direct online exploration is unsafe and infeasible. Despite this requirement, the vast majority of dynamic treatment optimization studies use off-policy RL methods (e.g., Double Deep Q Networks (DDQN) or its variants) that are known to perform poorly in purely offline settings. Recent advances in offline RL, such as Conservative Q-Learning (CQL), offer a suitable alternative. But there remain challenges in adapting these approaches to real-world applications where suboptimal examples dominate the retrospective dataset and strict safety constraints need to be satisfied. In this work, we introduce a practical transition sampling approach to address action imbalance during offline RL training, and an intuitive heuristic to enforce hard constraints during policy execution. We provide theoretical analyses to show that our proposed approach would improve over CQL. We perform extensive experiments on two real-world tasks for diabetes and sepsis treatment optimization to compare performance of the proposed approach against prominent off-policy and offline RL baselines (DDQN and CQL). Across a range of principled and clinically relevant metrics, we show that our proposed approach enables substantial improvements in expected health outcomes and in consistency with relevant practice and safety guidelines

Kalman Filter Models for the Prediction of Individualised Thermal Work Strain

It is important to monitor and assess the physiological strain of individuals working in hot environments to avoid heat illness and performance degradation. The body core temperature (Tc) is a reliable indicator of thermal work strain. However, measuring Tc is invasive and often inconvenient and impractical for real-time monitoring of workers in high heat strain environments. Seeking a better solution, the main aim of the present study was to investigate the Kalman filter method to enable the estimation of heat strain from non-invasive measurements (heart rate (HR) and chest skin temperature (ST)) obtained ‘online’ via wearable body sensors. In particular, we developed two Kalman filter models. First, an extended Kalman filter (EFK) was implemented in a cubic state space modelling framework (HR versus Tc) with a stage-wise, autoregressive exogenous model (incorporating HR and ST) as the time update model. Under the second model, the online Kalman filter (OFK) approach builds up the time update equation depending only on the initial value of Tc and the latest value of the exogenous variables. Both models were trained and validated using data from laboratory- and outfield-based heat strain profiling studies in which subjects performed a high intensity military foot march. While both the EKF and OKF models provided satisfactory estimates of Tc, the results showed an overall superior performance of the OKF model (overall root mean square error, RMSE = 0.31°C) compared to the EKF model (RMSE = 0.45°C)

Characterization of oil palm Acyl-CoA-Binding proteins and correlation of their gene expression with oil synthesis

Acyl-CoA-binding proteins (ACBPs) are involved in binding and trafficking acyl-CoA esters in eukaryotic cells. ACBPs contain a well-conserved acyl-CoA-binding domain (ACBD). Their various functions have been characterized in the model plant Arabidopsis and, to a lesser extent, in rice. In this study, genome-wide detection and expression analysis of ACBPs were performed on Elaeis guineensis (oil palm), the most important oil crop in the world. Seven E. guineensis ACBPs were identified and classified into four groups according to their deduced amino acid domain organization. Phylogenetic analysis showed conservation of this family with other higher plants. All seven EgACBPs were expressed in most tissues while their differential expression suggests various functions in specific tissues. For example, EgACBP3 had high expression in inflorescences and stalks while ACBP1 showed strong expression in leaves. Because of the importance of E. guineensis as an oil crop, expression of EgACBPs was specifically examined during fruit development. EgACBP3 showed high expression throughout mesocarp development, while EgACBP1 had enhanced expression during rapid oil synthesis. In endosperm, both EgACBP1 and EgACBP3 exhibited increased expression during seed development. These results provide important information for further investigations on the biological functions of EgACBPs in various tissues and, in particular, their roles in oil synthesis

Multiple novel prostate cancer susceptibility signals identified by fine-mapping of known risk loci among Europeans

Genome-wide association studies (GWAS) have identified numerous common prostate cancer (PrCa) susceptibility loci. We have fine-mapped 64 GWAS regions known at the conclusion of the iCOGS study using large-scale genotyping and imputation in 25 723 PrCa cases and 26 274 controls of European ancestry. We detected evidence for multiple independent signals at 16 regions, 12 of which contained additional newly identified significant associations. A single signal comprising a spectrum of correlated variation was observed at 39 regions; 35 of which are now described by a novel more significantly associated lead SNP, while the originally reported variant remained as the lead SNP only in 4 regions. We also confirmed two association signals in Europeans that had been previously reported only in East-Asian GWAS. Based on statistical evidence and linkage disequilibrium (LD) structure, we have curated and narrowed down the list of the most likely candidate causal variants for each region. Functional annotation using data from ENCODE filtered for PrCa cell lines and eQTL analysis demonstrated significant enrichment for overlap with bio-features within this set. By incorporating the novel risk variants identified here alongside the refined data for existing association signals, we estimate that these loci now explain ∼38.9% of the familial relative risk of PrCa, an 8.9% improvement over the previously reported GWAS tag SNPs. This suggests that a significant fraction of the heritability of PrCa may have been hidden during the discovery phase of GWAS, in particular due to the presence of multiple independent signals within the same regio

Measuring performance on the Healthcare Access and Quality Index for 195 countries and territories and selected subnational locations: A systematic analysis from the Global Burden of Disease Study 2016

Background: A key component of achieving universal health coverage is ensuring that all populations have access to quality health care. Examining where gains have occurred or progress has faltered across and within countries is crucial to guiding decisions and strategies for future improvement. We used the Global Burden of Diseases, Injuries, and Risk Factors Study 2016 (GBD 2016) to assess personal health-care access and quality with the Healthcare Access and Quality (HAQ) Index for 195 countries and territories, as well as subnational locations in seven countries, from 1990 to 2016. Methods Drawing from established methods and updated estimates from GBD 2016, we used 32 causes from which death should not occur in the presence of effective care to approximate personal health-care access and quality by location and over time. To better isolate potential effects of personal health-care access and quality from underlying risk factor patterns, we risk-standardised cause-specific deaths due to non-cancers by location-year, replacing the local joint exposure of environmental and behavioural risks with the global level of exposure. Supported by the expansion of cancer registry data in GBD 2016, we used mortality-to-incidence ratios for cancers instead of risk-standardised death rates to provide a stronger signal of the effects of personal health care and access on cancer survival. We transformed each cause to a scale of 0-100, with 0 as the first percentile (worst) observed between 1990 and 2016, and 100 as the 99th percentile (best); we set these thresholds at the country level, and then applied them to subnational locations. We applied a principal components analysis to construct the HAQ Index using all scaled cause values, providing an overall score of 0-100 of personal health-care access and quality by location over time. We then compared HAQ Index levels and trends by quintiles on the Socio-demographic Index (SDI), a summary measure of overall development. As derived from the broader GBD study and other data sources, we examined relationships between national HAQ Index scores and potential correlates of performance, such as total health spending per capita. Findings In 2016, HAQ Index performance spanned from a high of 97\ub71 (95% UI 95\ub78-98\ub71) in Iceland, followed by 96\ub76 (94\ub79-97\ub79) in Norway and 96\ub71 (94\ub75-97\ub73) in the Netherlands, to values as low as 18\ub76 (13\ub71-24\ub74) in the Central African Republic, 19\ub70 (14\ub73-23\ub77) in Somalia, and 23\ub74 (20\ub72-26\ub78) in Guinea-Bissau. The pace of progress achieved between 1990 and 2016 varied, with markedly faster improvements occurring between 2000 and 2016 for many countries in sub-Saharan Africa and southeast Asia, whereas several countries in Latin America and elsewhere saw progress stagnate after experiencing considerable advances in the HAQ Index between 1990 and 2000. Striking subnational disparities emerged in personal health-care access and quality, with China and India having particularly large gaps between locations with the highest and lowest scores in 2016. In China, performance ranged from 91\ub75 (89\ub71-93\ub76) in Beijing to 48\ub70 (43\ub74-53\ub72) in Tibet (a 43\ub75-point difference), while India saw a 30\ub78-point disparity, from 64\ub78 (59\ub76-68\ub78) in Goa to 34\ub70 (30\ub73-38\ub71) in Assam. Japan recorded the smallest range in subnational HAQ performance in 2016 (a 4\ub78-point difference), whereas differences between subnational locations with the highest and lowest HAQ Index values were more than two times as high for the USA and three times as high for England. State-level gaps in the HAQ Index in Mexico somewhat narrowed from 1990 to 2016 (from a 20\ub79-point to 17\ub70-point difference), whereas in Brazil, disparities slightly increased across states during this time (a 17\ub72-point to 20\ub74-point difference). Performance on the HAQ Index showed strong linkages to overall development, with high and high-middle SDI countries generally having higher scores and faster gains for non-communicable diseases. Nonetheless, countries across the development spectrum saw substantial gains in some key health service areas from 2000 to 2016, most notably vaccine-preventable diseases. Overall, national performance on the HAQ Index was positively associated with higher levels of total health spending per capita, as well as health systems inputs, but these relationships were quite heterogeneous, particularly among low-to-middle SDI countries. Interpretation GBD 2016 provides a more detailed understanding of past success and current challenges in improving personal health-care access and quality worldwide. Despite substantial gains since 2000, many low-SDI and middle- SDI countries face considerable challenges unless heightened policy action and investments focus on advancing access to and quality of health care across key health services, especially non-communicable diseases. Stagnating or minimal improvements experienced by several low-middle to high-middle SDI countries could reflect the complexities of re-orienting both primary and secondary health-care services beyond the more limited foci of the Millennium Development Goals. Alongside initiatives to strengthen public health programmes, the pursuit of universal health coverage hinges upon improving both access and quality worldwide, and thus requires adopting a more comprehensive view-and subsequent provision-of quality health care for all populations

Measuring performance on the Healthcare Access and Quality Index for 195 countries and territories and selected subnational locations: A systematic analysis from the Global Burden of Disease Study 2016

Copyright © 2018 The Author(s). Published by Elsevier Ltd. Background A key component of achieving universal health coverage is ensuring that all populations have access to quality health care. Examining where gains have occurred or progress has faltered across and within countries is crucial to guiding decisions and strategies for future improvement. We used the Global Burden of Diseases, Injuries, and Risk Factors Study 2016 (GBD 2016) to assess personal health-care access and quality with the Healthcare Access and Quality (HAQ) Index for 195 countries and territories, as well as subnational locations in seven countries, from 1990 to 2016. Methods Drawing from established methods and updated estimates from GBD 2016, we used 32 causes from which death should not occur in the presence of effective care to approximate personal health-care access and quality by location and over time. To better isolate potential effects of personal health-care access and quality from underlying risk factor patterns, we risk-standardised cause-specific deaths due to non-cancers by location-year, replacing the local joint exposure of environmental and behavioural risks with the global level of exposure. Supported by the expansion of cancer registry data in GBD 2016, we used mortality-to-incidence ratios for cancers instead of risk-standardised death rates to provide a stronger signal of the effects of personal health care and access on cancer survival. We transformed each cause to a scale of 0-100, with 0 as the first percentile (worst) observed between 1990 and 2016, and 100 as the 99th percentile (best); we set these thresholds at the country level, and then applied them to subnational locations. We applied a principal components analysis to construct the HAQ Index using all scaled cause values, providing an overall score of 0-100 of personal health-care access and quality by location over time. We then compared HAQ Index levels and trends by quintiles on the Socio-demographic Index (SDI), a summary measure of overall development. As derived from the broader GBD study and other data sources, we examined relationships between national HAQ Index scores and potential correlates of performance, such as total health spending per capita. Findings In 2016, HAQ Index performance spanned from a high of 97·1 (95% UI 95·8-98·1) in Iceland, followed by 96·6 (94·9-97·9) in Norway and 96·1 (94·5-97·3) in the Netherlands, to values as low as 18·6 (13·1-24·4) in the Central African Republic, 19·0 (14·3-23·7) in Somalia, and 23·4 (20·2-26·8) in Guinea-Bissau. The pace of progress achieved between 1990 and 2016 varied, with markedly faster improvements occurring between 2000 and 2016 for many countries in sub-Saharan Africa and southeast Asia, whereas several countries in Latin America and elsewhere saw progress stagnate after experiencing considerable advances in the HAQ Index between 1990 and 2000. Striking subnational disparities emerged in personal health-care access and quality, with China and India having particularly large gaps between locations with the highest and lowest scores in 2016. In China, performance ranged from 91·5 (89·1-93·6) in Beijing to 48·0 (43·4-53·2) in Tibet (a 43·5-point difference), while India saw a 30·8-point disparity, from 64·8 (59·6-68·8) in Goa to 34·0 (30·3-38·1) in Assam. Japan recorded the smallest range in subnational HAQ performance in 2016 (a 4·8-point difference), whereas differences between subnational locations with the highest and lowest HAQ Index values were more than two times as high for the USA and three times as high for England. State-level gaps in the HAQ Index in Mexico somewhat narrowed from 1990 to 2016 (from a 20·9-point to 17·0-point difference), whereas in Brazil, disparities slightly increased across states during this time (a 17·2-point to 20·4-point difference). Performance on the HAQ Index showed strong linkages to overall development, with high and high-middle SDI countries generally having higher scores and faster gains for non-communicable diseases. Nonetheless, countries across the development spectrum saw substantial gains in some key health service areas from 2000 to 2016, most notably vaccine-preventable diseases. Overall, national performance on the HAQ Index was positively associated with higher levels of total health spending per capita, as well as health systems inputs, but these relationships were quite heterogeneous, particularly among low-to-middle SDI countries. Interpretation GBD 2016 provides a more detailed understanding of past success and current challenges in improving personal health-care access and quality worldwide. Despite substantial gains since 2000, many low-SDI and middle- SDI countries face considerable challenges unless heightened policy action and investments focus on advancing access to and quality of health care across key health services, especially non-communicable diseases. Stagnating or minimal improvements experienced by several low-middle to high-middle SDI countries could reflect the complexities of re-orienting both primary and secondary health-care services beyond the more limited foci of the Millennium Development Goals. Alongside initiatives to strengthen public health programmes, the pursuit of universal health coverage hinges upon improving both access and quality worldwide, and thus requires adopting a more comprehensive view - and subsequent provision - of quality health care for all populations

Height and body-mass index trajectories of school-aged children and adolescents from 1985 to 2019 in 200 countries and territories: a pooled analysis of 2181 population-based studies with 65 million participants

Summary Background Comparable global data on health and nutrition of school-aged children and adolescents are scarce. We aimed to estimate age trajectories and time trends in mean height and mean body-mass index (BMI), which measures weight gain beyond what is expected from height gain, for school-aged children and adolescents. Methods For this pooled analysis, we used a database of cardiometabolic risk factors collated by the Non-Communicable Disease Risk Factor Collaboration. We applied a Bayesian hierarchical model to estimate trends from 1985 to 2019 in mean height and mean BMI in 1-year age groups for ages 5–19 years. The model allowed for non-linear changes over time in mean height and mean BMI and for non-linear changes with age of children and adolescents, including periods of rapid growth during adolescence. Findings We pooled data from 2181 population-based studies, with measurements of height and weight in 65 million participants in 200 countries and territories. In 2019, we estimated a difference of 20 cm or higher in mean height of 19-year-old adolescents between countries with the tallest populations (the Netherlands, Montenegro, Estonia, and Bosnia and Herzegovina for boys; and the Netherlands, Montenegro, Denmark, and Iceland for girls) and those with the shortest populations (Timor-Leste, Laos, Solomon Islands, and Papua New Guinea for boys; and Guatemala, Bangladesh, Nepal, and Timor-Leste for girls). In the same year, the difference between the highest mean BMI (in Pacific island countries, Kuwait, Bahrain, The Bahamas, Chile, the USA, and New Zealand for both boys and girls and in South Africa for girls) and lowest mean BMI (in India, Bangladesh, Timor-Leste, Ethiopia, and Chad for boys and girls; and in Japan and Romania for girls) was approximately 9–10 kg/m2. In some countries, children aged 5 years started with healthier height or BMI than the global median and, in some cases, as healthy as the best performing countries, but they became progressively less healthy compared with their comparators as they grew older by not growing as tall (eg, boys in Austria and Barbados, and girls in Belgium and Puerto Rico) or gaining too much weight for their height (eg, girls and boys in Kuwait, Bahrain, Fiji, Jamaica, and Mexico; and girls in South Africa and New Zealand). In other countries, growing children overtook the height of their comparators (eg, Latvia, Czech Republic, Morocco, and Iran) or curbed their weight gain (eg, Italy, France, and Croatia) in late childhood and adolescence. When changes in both height and BMI were considered, girls in South Korea, Vietnam, Saudi Arabia, Turkey, and some central Asian countries (eg, Armenia and Azerbaijan), and boys in central and western Europe (eg, Portugal, Denmark, Poland, and Montenegro) had the healthiest changes in anthropometric status over the past 3·5 decades because, compared with children and adolescents in other countries, they had a much larger gain in height than they did in BMI. The unhealthiest changes—gaining too little height, too much weight for their height compared with children in other countries, or both—occurred in many countries in sub-Saharan Africa, New Zealand, and the USA for boys and girls; in Malaysia and some Pacific island nations for boys; and in Mexico for girls. Interpretation The height and BMI trajectories over age and time of school-aged children and adolescents are highly variable across countries, which indicates heterogeneous nutritional quality and lifelong health advantages and risks

Global variations in diabetes mellitus based on fasting glucose and haemogloblin A1c

Fasting plasma glucose (FPG) and haemoglobin A1c (HbA1c) are both used to diagnose diabetes, but may identify different people as having diabetes. We used data from 117 population-based studies and quantified, in different world regions, the prevalence of diagnosed diabetes, and whether those who were previously undiagnosed and detected as having diabetes in survey screening had elevated FPG, HbA1c, or both. We developed prediction equations for estimating the probability that a person without previously diagnosed diabetes, and at a specific level of FPG, had elevated HbA1c, and vice versa. The age-standardised proportion of diabetes that was previously undiagnosed, and detected in survey screening, ranged from 30% in the high-income western region to 66% in south Asia. Among those with screen-detected diabetes with either test, the agestandardised proportion who had elevated levels of both FPG and HbA1c was 29-39% across regions; the remainder had discordant elevation of FPG or HbA1c. In most low- and middle-income regions, isolated elevated HbA1c more common than isolated elevated FPG. In these regions, the use of FPG alone may delay diabetes diagnosis and underestimate diabetes prevalence. Our prediction equations help allocate finite resources for measuring HbA1c to reduce the global gap in diabetes diagnosis and surveillance.peer-reviewe