Noisy Optimization: Convergence with a Fixed Number of Resamplings

It is known that evolution strategies in continuous domains might not converge in the presence of noise. It is also known that, under mild assumptions, and using an increasing number of resamplings, one can mitigate the effect of additive noise and recover convergence. We show new sufficient conditions for the convergence of an evolutionary algorithm with constant number of resamplings; in particular, we get fast rates (log-linear convergence) provided that the variance decreases around the optimum slightly faster than in the so-called multiplicative noise model. Keywords: Noisy optimization, evolutionary algorithm, theory.Comment: EvoStar (2014

Warm-Start AlphaZero Self-Play Search Enhancements

Recently, AlphaZero has achieved landmark results in deep reinforcement learning, by providing a single self-play architecture that learned three different games at super human level. AlphaZero is a large and complicated system with many parameters, and success requires much compute power and fine-tuning. Reproducing results in other games is a challenge, and many researchers are looking for ways to improve results while reducing computational demands. AlphaZero's design is purely based on self-play and makes no use of labeled expert data ordomain specific enhancements; it is designed to learn from scratch. We propose a novel approach to deal with this cold-start problem by employing simple search enhancements at the beginning phase of self-play training, namely Rollout, Rapid Action Value Estimate (RAVE) and dynamically weighted combinations of these with the neural network, and Rolling Horizon Evolutionary Algorithms (RHEA). Our experiments indicate that most of these enhancements improve the performance of their baseline player in three different (small) board games, with especially RAVE based variants playing strongly

A Model of Oxidative Stress Management: Moderation of Carbohydrate Metabolizing Enzymes in SOD1-Null Drosophila melanogaster

The response to oxidative stress involves numerous genes and mutations in these genes often manifest in pleiotropic ways that presumably reflect perturbations in ROS-mediated physiology. The Drosophila melanogaster SOD1-null allele (cSODn108) is proposed to result in oxidative stress by preventing superoxide breakdown. In SOD1-null flies, oxidative stress management is thought to be reliant on the glutathione-dependent antioxidants that utilize NADPH to cycle between reduced and oxidized form. Previous studies suggest that SOD1-null Drosophila rely on lipid catabolism for energy rather than carbohydrate metabolism. We tested these connections by comparing the activity of carbohydrate metabolizing enzymes, lipid and triglyceride concentration, and steady state NADPH:NADP+ in SOD1-null and control transgenic rescue flies. We find a negative shift in the activity of carbohydrate metabolizing enzymes in SOD1-nulls and the NADP+-reducing enzymes were found to have significantly lower activity than the other enzymes assayed. Little evidence for the catabolism of lipids as preferential energy source was found, as the concentration of lipids and triglycerides were not significantly lower in SOD1-nulls compared with controls. Using a starvation assay to impact lipids and triglycerides, we found that lipids were indeed depleted in both genotypes when under starvation stress, suggesting that oxidative damage was not preventing the catabolism of lipids in SOD1-null flies. Remarkably, SOD1-nulls were also found to be relatively resistant to starvation. Age profiles of enzyme activity, triglyceride and lipid concentration indicates that the trends observed are consistent over the average lifespan of the SOD1-nulls. Based on our results, we propose a model of physiological response in which organisms under oxidative stress limit the production of ROS through the down-regulation of carbohydrate metabolism in order to moderate the products exiting the electron transport chain

Adaptive preconditioning in neurological diseases -­ therapeutic insights from proteostatic perturbations

International audienceIn neurological disorders, both acute and chronic neural stress can disrupt cellular proteostasis, resulting in the generation of pathological protein. However in most cases, neurons adapt to these proteostatic perturbations by activating a range of cellular protective and repair responses, thus maintaining cell function. These interconnected adaptive mechanisms comprise a 'proteostasis network' and include the unfolded protein response, the ubiquitin proteasome system and autophagy. Interestingly, several recent studies have shown that these adaptive responses can be stimulated by preconditioning treatments, which confer resistance to a subsequent toxic challenge - the phenomenon known as hormesis. In this review we discuss the impact of adaptive stress responses stimulated in diverse human neuropathologies including Parkinson´s disease, Wolfram syndrome, brain ischemia, and brain cancer. Further, we examine how these responses - and the molecular pathways they recruit - might be exploited for therapeutic gai

Monte-Carlo Tree Search in Poker using Expected Reward Distributions

Abstract. We investigate the use of Monte-Carlo Tree Search (MCTS) within the field of computer Poker, more specifically No-Limit Texas Hold’em. The hidden information in Poker results in so called miximax game trees where opponent decision nodes have to be modeled as chance nodes. The probability distribution in these nodes is modeled by an opponent model that predicts the actions of the opponents. We propose a modification of the standard MCTS selection and backpropagation strategies that explicitly model and exploit the uncertainty of sampled expected values. The new strategies are evaluated as a part of a complete Poker bot that is, to the best of our knowledge, the first exploiting no-limit Texas Hold’em bot that can play at a reasonable level in games of more than two players.