Connecting stochastic optimal control and reinforcement learning

In this paper the connection between stochastic optimal control and reinforcement learning is investigated. Our main motivation is to apply importance sampling to sampling rare events which can be reformulated as an optimal control problem. By using a parameterised approach the optimal control problem becomes a stochastic optimization problem which still raises some open questions regarding how to tackle the scalability to high-dimensional problems and how to deal with the intrinsic metastability of the system. To explore new methods we link the optimal control problem to reinforcement learning since both share the same underlying framework, namely a Markov Decision Process (MDP). For the optimal control problem we show how the MDP can be formulated. In addition we discuss how the stochastic optimal control problem can be interpreted in the framework of reinforcement learning. At the end of the article we present the application of two different reinforcement learning algorithms to the optimal control problem and a comparison of the advantages and disadvantages of the two algorithms

Animal Studies Journal 2015 4 (1): Cover Pages, Table of Contents, Notes on Contributors and Editorial

Cover pages, table of contents, contributor biographies and editorial for Animal Studies Journal Vol. 4 No.1, 2015. Guest editor - Colin Salter

Diagnostic Yield of Exome Sequencing in Fetuses With Multisystem Malformations: Systematic Review and Meta-analysis

Objective: To determine the diagnostic yield of exome sequencing (ES) above that of chromosomal microarray analysis (CMA) or karyotyping in fetuses with multisystem structural anomalies (at least two major anomalies in different anatomical systems). Method: This was a systematic review conducted in accordance with PRISMA guidelines. Searching PubMed, Web of Knowledge and Cochrane database, we identified studies describing ES, whole-genome and/or next-generation sequencing in fetuses with multisystem malformations. Included were observational studies involving five or more eligible fetuses. A fetus was eligible for inclusion if it had at least two major anomalies of different anatomical systems and a negative CMA or karyotyping result. Only positive variants classified as likely pathogenic or pathogenic determined to be causative of the fetal phenotype were considered. A negative CMA or karyotype result was treated as the reference standard. The diagnostic yield of the primary outcome was calculated by single-proportion analysis using random-effects modeling. A subgroup analysis was performed to compare the diagnostic yield of the solo approach (fetus alone sequenced) with that of the trio approach (fetus and both parents sequenced). Results: Seventeen articles with data on ES diagnostic yield, including 694 individuals with multisystem malformations, were identified. Overall, a pathogenic or likely pathogenic variant potentially causative of the fetal phenotype was found in 213 fetuses, giving a 33% (95% CI, 27–40%) incremental yield of ES. A stratified analysis showed similar diagnostic yields of ES using the solo approach (30%; 95% CI, 11–52%) and the trio approach (35%; 95% CI, 26–44%). Conclusions: ES applied in fetuses with multisystem structural anomalies was able to identify a potentially causative gene when CMA or karyotyping had failed to do so in an additional one-third of cases. No differences were observed between the solo and trio approaches for ES. © 2022 International Society of Ultrasound in Obstetrics and Gynecology

Elucidating the mystery of the tripartite symbiosis plant – mycorrhizal fungi – dark septate endophytes

Non-Peer ReviewedThis study provides information on the tripartite symbiotic relationships formed by arbuscular mycorrhizal fungi (AMF) and dark septate endophytes (DSE) in crops growing in the semiarid region of the Canadian Prairie. We found the symbiotic root systems of wheat, pea, chickpea and lentil to be morphologically distinct. The relationship between DSE and AMF abundance in roots ranged from negative in lentil to positive in wheat

Racial/ethnic inequities in the associations of allostatic load with all-cause and cardiovascular-specific mortality risk in U.S. adults

Non-Hispanic blacks have higher mortality rates than non-Hispanic whites whereas Hispanics have similar or lower mortality rates than non-Hispanic blacks and whites despite Hispanics' lower education and access to health insurance coverage. This study examines whether allostatic load, a proxy for cumulative biological risk, is associated with all-cause and cardiovascular (CVD)-specific mortality risks in US adults; and whether these associations vary with race/ethnicity and further with age, sex and education across racial/ethnic groups. Data from the third National Health and Nutritional Examination Survey (NHANES III, 1988-1994) and the 2015 Linked Mortality File were used for adults 25 years or older (n = 13,673 with 6,026 deaths). Cox proportional hazards regression was used to estimate the associations of allostatic load scores (2 and >= 3 relative to <= 1) with a) all-cause and b) CVD-specific mortality risk among NHANES III participants before and after controlling for selected characteristics. Allostatic load scores are associated with higher all-cause and CVD-specific mortality rates among U.S. adults aged 25 years or older, with stronger rates observed for CVD-specific mortality. All-cause mortality rates for each racial/ethnic group differed with age and education whereas for CVD-specific mortality rates, this difference was observed for sex. Our findings of high allostatic load scores associated with all-cause and CVD-specific mortality among US adults call attention to monitor conditions associated with the allostatic load's biomarkers to identify high-risk groups to help monitor social inequities in mortality risk, especially premature mortality

Socioeconomic Inequalities in Mortality Rates in Old Age in the World Health Organization Europe Region

Socioeconomic adversity is among the foremost fundamental causes of human suffering, and this is no less true in old age. Recent reports on socioeconomic inequalities in mortality rate in old age suggest that a low socioeconomic position continues to increase the risk of death even among the oldest old. We aimed to examine the evidence for socioeconomic mortality rate inequalities in old age, including information about associations with various indicators of socioeconomic position and for various geographic locations within the World Health Organization Region for Europe. The articles included in this review leave no doubt that inequalities in mortality rate by socioeconomic position persist into the oldest ages for both men and women in all countries for which information is available, although the relative risk measures observed were rarely higher than 2.00. Still, the available evidence base is heavily biased geographically, inasmuch as it is based largely on national studies from Nordic and Western European countries and local studies from urban areas in Southern Europe. This bias will hamper the design of European-wide policies to reduce inequalities in mortality rate. We call for a continuous update of the empiric evidence on socioeconomic inequalities in mortality rate

HPC-enabling technologies for high-fidelity combustion simulations

With the increase in computational power in the last decade and the forthcoming Exascale supercomputers, a new horizon in computational modelling and simulation is envisioned in combustion science. Considering the multiscale and multiphysics characteristics of turbulent reacting flows, combustion simulations are considered as one of the most computationally demanding applications running on cutting-edge supercomputers. Exascale computing opens new frontiers for the simulation of combustion systems as more realistic conditions can be achieved with high-fidelity methods. However, an efficient use of these computing architectures requires methodologies that can exploit all levels of parallelism. The efficient utilization of the next generation of supercomputers needs to be considered from a global perspective, that is, involving physical modelling and numerical methods with methodologies based on High-Performance Computing (HPC) and hardware architectures. This review introduces recent developments in numerical methods for large-eddy simulations (LES) and direct-numerical simulations (DNS) to simulate combustion systems, with focus on the computational performance and algorithmic capabilities. Due to the broad scope, a first section is devoted to describe the fundamentals of turbulent combustion, which is followed by a general description of state-of-the-art computational strategies for solving these problems. These applications require advanced HPC approaches to exploit modern supercomputers, which is addressed in the third section. The increasing complexity of new computing architectures, with tightly coupled CPUs and GPUs, as well as high levels of parallelism, requires new parallel models and algorithms exposing the required level of concurrency. Advances in terms of dynamic load balancing, vectorization, GPU acceleration and mesh adaptation have permitted to achieve highly-efficient combustion simulations with data-driven methods in HPC environments. Therefore, dedicated sections covering the use of high-order methods for reacting flows, integration of detailed chemistry and two-phase flows are addressed. Final remarks and directions of future work are given at the end. }The research leading to these results has received funding from the European Union’s Horizon 2020 Programme under the CoEC project, grant agreement No. 952181 and the CoE RAISE project grant agreement no. 951733.Peer ReviewedPostprint (published version