Warm-Start AlphaZero Self-Play Search Enhancements

Recently, AlphaZero has achieved landmark results in deep reinforcement learning, by providing a single self-play architecture that learned three different games at super human level. AlphaZero is a large and complicated system with many parameters, and success requires much compute power and fine-tuning. Reproducing results in other games is a challenge, and many researchers are looking for ways to improve results while reducing computational demands. AlphaZero's design is purely based on self-play and makes no use of labeled expert data ordomain specific enhancements; it is designed to learn from scratch. We propose a novel approach to deal with this cold-start problem by employing simple search enhancements at the beginning phase of self-play training, namely Rollout, Rapid Action Value Estimate (RAVE) and dynamically weighted combinations of these with the neural network, and Rolling Horizon Evolutionary Algorithms (RHEA). Our experiments indicate that most of these enhancements improve the performance of their baseline player in three different (small) board games, with especially RAVE based variants playing strongly

BoostingTree: parallel selection of weak learners in boosting, with application to ranking

Boosting algorithms have been found successful in many areas of machine learning and, in particular, in ranking. For typical classes of weak learners used in boosting (such as decision stumps or trees), a large feature space can slow down the training, while a long sequence of weak hypotheses combined by boosting can result in a computationally expensive model. In this paper we propose a strategy that builds several sequences of weak hypotheses in parallel, and extends the ones that are likely to yield a good model. The weak hypothesis sequences are arranged in a boosting tree, and new weak hypotheses are added to promising nodes (both leaves and inner nodes) of the tree using some randomized method. Theoretical results show that the proposed algorithm asymptotically achieves the performance of the base boosting algorithm applied. Experiments are provided in ranking web documents and move ordering in chess, and the results indicate that the new strategy yields better performance when the length of the sequence is limited, and converges to similar performance as the original boosting algorithms otherwise. © 2013 The Author(s)

Replication for Web Hosting Systems

Replication is a well-known technique to improve the accessibility of Web sites. It generally offers reduced client latencies and increases a site’s availability. However, applying replication techniques is not trivial, and various Content Delivery Networks (CDNs) have been created to facilitate replication for digital content providers. Th

Interrogating and Predicting Tolerated Sequence Diversity in Protein Folds: Application to E. elaterium Trypsin Inhibitor-II Cystine-Knot Miniprotein

Cystine-knot miniproteins (knottins) are promising molecular scaffolds for protein engineering applications. Members of the knottin family have multiple loops capable of displaying conformationally constrained polypeptides for molecular recognition. While previous studies have illustrated the potential of engineering knottins with modified loop sequences, a thorough exploration into the tolerated loop lengths and sequence space of a knottin scaffold has not been performed. In this work, we used the Ecballium elaterium trypsin inhibitor II (EETI) as a model member of the knottin family and constructed libraries of EETI loop-substituted variants with diversity in both amino acid sequence and loop length. Using yeast surface display, we isolated properly folded EETI loop-substituted clones and applied sequence analysis tools to assess the tolerated diversity of both amino acid sequence and loop length. In addition, we used covariance analysis to study the relationships between individual positions in the substituted loops, based on the expectation that correlated amino acid substitutions will occur between interacting residue pairs. We then used the results of our sequence and covariance analyses to successfully predict loop sequences that facilitated proper folding of the knottin when substituted into EETI loop 3. The sequence trends we observed in properly folded EETI loop-substituted clones will be useful for guiding future protein engineering efforts with this knottin scaffold. Furthermore, our findings demonstrate that the combination of directed evolution with sequence and covariance analyses can be a powerful tool for rational protein engineering

The European Solar Telescope

The European Solar Telescope (EST) is a project aimed at studying the magnetic connectivity of the solar atmosphere, from the deep photosphere to the upper chromosphere. Its design combines the knowledge and expertise gathered by the European solar physics community during the construction and operation of state-of-the-art solar telescopes operating in visible and near-infrared wavelengths: the Swedish 1m Solar Telescope, the German Vacuum Tower Telescope and GREGOR, the French Télescope Héliographique pour l'Étude du Magnétisme et des Instabilités Solaires, and the Dutch Open Telescope. With its 4.2 m primary mirror and an open configuration, EST will become the most powerful European ground-based facility to study the Sun in the coming decades in the visible and near-infrared bands. EST uses the most innovative technological advances: the first adaptive secondary mirror ever used in a solar telescope, a complex multi-conjugate adaptive optics with deformable mirrors that form part of the optical design in a natural way, a polarimetrically compensated telescope design that eliminates the complex temporal variation and wavelength dependence of the telescope Mueller matrix, and an instrument suite containing several (etalon-based) tunable imaging spectropolarimeters and several integral field unit spectropolarimeters. This publication summarises some fundamental science questions that can be addressed with the telescope, together with a complete description of its major subsystems

The European Solar Telescope

The European Solar Telescope (EST) is a project aimed at studying the magnetic connectivity of the solar atmosphere, from the deep photosphere to the upper chromosphere. Its design combines the knowledge and expertise gathered by the European solar physics community during the construction and operation of state-of-the-art solar telescopes operating in visible and near-infrared wavelengths: the Swedish 1m Solar Telescope, the German Vacuum Tower Telescope and GREGOR, the French Télescope Héliographique pour l’Étude du Magnétisme et des Instabilités Solaires, and the Dutch Open Telescope. With its 4.2 m primary mirror and an open configuration, EST will become the most powerful European ground-based facility to study the Sun in the coming decades in the visible and near-infrared bands. EST uses the most innovative technological advances: the first adaptive secondary mirror ever used in a solar telescope, a complex multi-conjugate adaptive optics with deformable mirrors that form part of the optical design in a natural way, a polarimetrically compensated telescope design that eliminates the complex temporal variation and wavelength dependence of the telescope Mueller matrix, and an instrument suite containing several (etalon-based) tunable imaging spectropolarimeters and several integral field unit spectropolarimeters. This publication summarises some fundamental science questions that can be addressed with the telescope, together with a complete description of its major subsystems

DCMA, yet another derandomization in Covariance-Matrix-Adaptation

International audienceIn a preliminary part of this paper, we analyze the neces- sity of randomness in evolution strategies. We conclude to the necessity of ”continuous”-randomness, but with a much more limited use of randomness than what is commonly used in evolution strategies. We then apply these results to CMA- ES, a famous evolution strategy already based on the idea of derandomization, which uses random independent Gaus- sian mutations. We here replace these random independent Gaussian mutations by a quasi-random sample. The mod- ification is very easy to do, the modified algorithm is com- putationally more efficient and its convergence is faster in terms of the number of iterates for a given precision