Constrained Model-Free Reinforcement Learning for Process Optimization

Reinforcement learning (RL) is a control approach that can handle nonlinear stochastic optimal control problems. However, despite the promise exhibited, RL has yet to see marked translation to industrial practice primarily due to its inability to satisfy state constraints. In this work we aim to address this challenge. We propose an 'oracle'-assisted constrained Q-learning algorithm that guarantees the satisfaction of joint chance constraints with a high probability, which is crucial for safety critical tasks. To achieve this, constraint tightening (backoffs) are introduced and adjusted using Broyden's method, hence making them self-tuned. This results in a general methodology that can be imbued into approximate dynamic programming-based algorithms to ensure constraint satisfaction with high probability. Finally, we present case studies that analyze the performance of the proposed approach and compare this algorithm with model predictive control (MPC). The favorable performance of this algorithm signifies a step toward the incorporation of RL into real world optimization and control of engineering systems, where constraints are essential in ensuring safety

Potential of a sequence-based antigenic distance measure to indicate equine influenza vaccine strain efficacy

The calculation of pepitope values, a sequence-based measure of antigenic distance between strains, was developed for human influenza. The potential to apply the pepitope value to equine influenza vaccine strain selection was assessed. There was a negative correlation between pepitope value and vaccine efficacy for pairs of vaccine and challenge strains used in cross-protection studies in ponies that just reached statistical significance (p = 0.046) only if one pair of viruses was excluded from the analysis. Thus the pepitope value has potential to provide additional data to consider in the decision-making process for updating equine influenza vaccine strains. However, further work is required to define the epitopes of the equine H3N8 haemagglutinin protein recognised by equine antibodies, which could lead to refinement of the pepitope value calculation. Furthermore, other factors such as vaccine potency and virulence of circulating strains may also influence vaccine efficacy

On the predictability of emerging market sovereign credit spreads

This paper examines the quarter-ahead out-of-sample predictability of Brazil, Mexico, the Philippines and Turkey credit spreads before and after the Lehman Brothers’ default. A model based on the country-specific credit spread curve factors predicts no better than the random walk and slope regression benchmarks. Model extensions with the global yield curve factors and with both global and domestic uncertainty indicators notably outperform both benchmarks post-Lehman. The finding that bond prices better reflect fundamental information after the Lehman Brothers’ failure indicates that this landmark of the recent global financial crisis had wake-up call effects on emerging market bond investors

Population neuroscience : challenges and opportunities for psychiatric research in low- and middle-income countries

Objective: Population neuroscience is an emerging field that combines epidemiology and neuroscience to study how genes and the environment shape typical and atypical brain functioning. The objective of this study was to review key studies on population neuroscience from low- and middle-income countries (LMICs) and to identify potential gaps vis-a`-vis studies conducted in highincome countries. Methods: We conducted a systematic review to search for longitudinal cohort studies investigating the development of psychiatric disorders in children and adolescents in LMICs. We performed an electronic search in the EMBASE and MEDLINE databases from inception to July 5th, 2019. Results: We found six cohorts from four countries that met our search criteria: three cohorts from Brazil, one from China, one from South Africa, and one from Mauritius. Relevant examples of findings from these studies are reported. Conclusion: Our results demonstrate the impact of the valuable science output these cohort designs promote, allowing LMICs to have a share in frontline global psychiatry research. National and international funding agencies should invest in LMIC population neuroscience in order to promote replication and generalization of research from high-income countries

Non-Diversifiable Volatility Risk and Risk Premiums at Earnings Announcements

This study seeks to determine whether earnings announcements pose non-diversifiable volatility risk that commands a risk premium. We find that investors anticipate some earnings announcements to convey news that increases market return volatility and pay a premium to hedge this non-diversifiable risk. In particular, we find evidence of risk premiums embedded in prices of firms' traded options that are significantly positively associated with the extent to which the firms' earnings announcements pose non-diversifiable volatility risk. In addition, we find that volatility risk premiums are concentrated among bellwether firms and result in predictable variation in option straddle returns around earnings announcements. Taken together, our findings show that some earnings announcements pose non-diversifiable volatility risk that commands a risk premium

Counter‐Credit‐Risk Yield Spreads: A Puzzle in China's Corporate Bond Market

yesIn this paper, using China’s risk-free and corporate zero yields together with aggregate credit risk measures and various control variables from 2006 to 2013, we document a puzzle of counter-credit-risk corporate yield spreads. We interpret this puzzle as a symptom of the immaturity of China’s credit bond market, which reveals a distorted pricing mechanism latent in the fundamental of this market. We also find interesting results about relationships between corporate yield spreads and interest rates as well as risk premia and the stock index, and these results are somewhat attributed to this puzzle

Ultrahigh-current-density niobium disulfide catalysts for hydrogen evolution

Metallic transition metal dichalcogenides (TMDs)1???8 are good catalysts for the hydrogen evolution reaction (HER). The overpotential and Tafel slope values of metallic phases and edges9 of two-dimensional (2D) TMDs approach those of Pt. However, the overall current density of 2D TMD catalysts remains orders of magnitude lower (~10???100 mA cm???2) than industrial Pt and Ir electrolysers (>1,000 mA cm???2)10,11. Here, we report the synthesis of the metallic 2H phase of niobium disulfide with additional niobium (2H Nb1+xS2, where x is ~0.35)12 as a HER catalyst with current densities of >5,000 mA cm???2 at ~420 mV versus a reversible hydrogen electrode. We find the exchange current density at 0 V for 2H Nb1.35S2 to be ~0.8 mA cm???2, corresponding to a turnover frequency of ~0.2 s???1. We demonstrate an electrolyser based on a 2H Nb1+ xS2 cathode that can generate current densities of 1,000 mA cm???2. Our theoretical results reveal that 2H Nb1+ xS2 with Nb-terminated surface has free energy for hydrogen adsorption that is close to thermoneutral, facilitating HER. Therefore, 2H Nb1+ xS2 could be a viable catalyst for practical electrolysers

Correlation of financial markets in times of crisis

Using the eigenvalues and eigenvectors of correlations matrices of some of the main financial market indices in the world, we show that high volatility of markets is directly linked with strong correlations between them. This means that markets tend to behave as one during great crashes. In order to do so, we investigate several financial market crises that occurred in the years 1987 (Black Monday), 1989 (Russian crisis), 2001 (Burst of the dot-com bubble and September 11), and 2008 (Subprime Mortgage Crisis), which mark some of the largest downturns of financial markets in the last three decades.Comment: 33 pages, 46 figure

Comparative ICE Genomics: Insights into the Evolution of the SXT/R391 Family of ICEs

Integrating and conjugative elements (ICEs) are one of the three principal types of self-transmissible mobile genetic elements in bacteria. ICEs, like plasmids, transfer via conjugation; but unlike plasmids and similar to many phages, these elements integrate into and replicate along with the host chromosome. Members of the SXT/R391 family of ICEs have been isolated from several species of gram-negative bacteria, including Vibrio cholerae, the cause of cholera, where they have been important vectors for disseminating genes conferring resistance to antibiotics. Here we developed a plasmid-based system to capture and isolate SXT/R391 ICEs for sequencing. Comparative analyses of the genomes of 13 SXT/R391 ICEs derived from diverse hosts and locations revealed that they contain 52 perfectly syntenic and nearly identical core genes that serve as a scaffold capable of mobilizing an array of variable DNA. Furthermore, selection pressure to maintain ICE mobility appears to have restricted insertions of variable DNA into intergenic sites that do not interrupt core functions. The variable genes confer diverse element-specific phenotypes, such as resistance to antibiotics. Functional analysis of a set of deletion mutants revealed that less than half of the conserved core genes are required for ICE mobility; the functions of most of the dispensable core genes are unknown. Several lines of evidence suggest that there has been extensive recombination between SXT/R391 ICEs, resulting in re-assortment of their respective variable gene content. Furthermore, our analyses suggest that there may be a network of phylogenetic relationships among sequences found in all types of mobile genetic elements