Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents

Evolution strategies (ES) are a family of black-box optimization algorithms able to train deep neural networks roughly as well as Q-learning and policy gradient methods on challenging deep reinforcement learning (RL) problems, but are much faster (e.g. hours vs. days) because they parallelize better. However, many RL problems require directed exploration because they have reward functions that are sparse or deceptive (i.e. contain local optima), and it is unknown how to encourage such exploration with ES. Here we show that algorithms that have been invented to promote directed exploration in small-scale evolved neural networks via populations of exploring agents, specifically novelty search (NS) and quality diversity (QD) algorithms, can be hybridized with ES to improve its performance on sparse or deceptive deep RL tasks, while retaining scalability. Our experiments confirm that the resultant new algorithms, NS-ES and two QD algorithms, NSR-ES and NSRA-ES, avoid local optima encountered by ES to achieve higher performance on Atari and simulated robots learning to walk around a deceptive trap. This paper thus introduces a family of fast, scalable algorithms for reinforcement learning that are capable of directed exploration. It also adds this new family of exploration algorithms to the RL toolbox and raises the interesting possibility that analogous algorithms with multiple simultaneous paths of exploration might also combine well with existing RL algorithms outside ES

Bioactivity of Different Chemotypes of Oregano Essential Oil against the Blowfly Calliphora vomitoria Vector of Foodborne Pathogens

Blowflies play a substantial role as vectors of microorganisms, including human pathogens. The control of these insect pests is an important aspect of the prevention of foodborne diseases, which represent a significant public health threat worldwide. Among aromatic plants, spices essential oils (EOs) are the most suitable to protect food from insect pests. In this study, we determined the chemical composition of three oregano EOs and assessed their toxicity and deterrence to oviposition against the blowfly Calliphora vomitoria L. The chemical analyses showed that the EOs belonged to three chemotypes: one with a prevalence of carvacrol, the carvacrol chemotype (CC; carvacrol, 81.5%), and two with a prevalence of thymol, the thymol/p-cymene and thymol/-terpinene chemotypes(TCC and TTC; thymol, 43.8, and 36.7%, respectively). The bioassays showed that although all the three EOs chemotypes are able to exert a toxic activity against C. vomitoria adults (LD50 from 0.14 to 0.31 L insect1) and eggs (LC50 from 0.008 to 0.038 L cm2) as well as deter the oviposition (Oviposition Activity Index, OAI, from 0.40 0.04 to 0.87 0.02), the bioactivity of oregano EOs significantly varies among the chemotypes, with the thymol-rich EOs (TCC and TTC) overall demonstrating more effectiveness than the carvacrol-rich (CC) EO

Reduced mitogenic stimulation of human lymphocytes by extremely low frequency electromagnetic fields

AbstractBlastogenesis of human peripheral blood lymphocytes stimulated in vitro by non-specific mitogens (PHA, ConA, PWM) upon exposure to extremely low frequency EMF has been studied. Different frequencies of square waveforms have been used. PHA-stimulation resulted in strong inhibitions as measured by [3H]thymidine incorporation. A frequency window (3–50 Hz) within which ConA-induced blastogenesis was significantly inhibited has been individuated. The mitogenic effect of PWM was significantly affected only at 3 Hz

Evolution of Noninvasive Mechanical Ventilation Use: A Cohort Study Among Italian PICUs

To assess how clinical practice of noninvasive ventilation has evolved in the Italian PICUs

Compton sources for the observation of elastic photon-photon scattering events

We present the design of a photon-photon collider based on conventional Compton gamma sources for the observation of elastic \u3b3\u3b3 scattering. Two symmetric electron beams, generated by photocathodes and accelerated in linacs, produce two primary gamma rays through Compton backscattering with two high energy lasers. The elastic photon-photon scattering is analyzed by start-to-end simulations from the photocathodes to the detector. A new Monte Carlo code has been developed ad hoc for the counting of the QED events. Realistic numbers of the secondary gamma yield, obtained by using the parameters of existing or approved Compton devices, a discussion of the feasibility of the experiment and of the nature of the background are presented

Renal parenchymal resistance in patients with biopsy proven glomerulonephritis: correlation with histological findings

Renal Doppler ultrasound is increasingly used in nephrology for the evaluation of renovascular disease, allograft dysfunction, and chronic nephropathies. We compared intrarenal hemodynamic parameters to biopsy findings of glomerular sclerosis, tubular atrophy, interstitial fibrosis, crescents, arteriolosclerosis, and clinical variables in 100 patients. A positive correlation exists between renal function and percentage of glomerular sclerosis (P <0.01, r = 0.26), conversely a negative correlation exists between glomerular filtrate rate and percentage of glomerular sclerosis(P <0.0001, r = -0.35). The percentage of glomerular sclerosis correlate positively with pulsatile index (PI) (P <0.05, r = 0.21) and renal resistive index (RI) (P <0.05, r = 0.20). The percentage of crescents correlates positively with PI(P <0.05, r = 0.21) and RI (P <0.05, r = 0.20). Classifying arteriolosclerosis in four groups according to a severity scale, from absence to severe, PI (P <0.05) and RI (P <0.01) were significantly different. In the post hoc analysis, the median values of PI and RI are significantly different in patients with severe arteriolosclerosis than others. Ultrasound examination is a non-invasive diagnostic technique used on patients with suspected or established renal disease. Our study shows a close correlation between kidney function, ultrasound parameters, and histological findings. Measurement of renal parenchymal resistance by ultrasound could be used in association with biopsy and glomerular function for the evaluation of renal damage in patients with glomerulonephritis