Behavioral repertoire learning in robotics

Behavioral Repertoire Learning in Robotics Antoine Cully ISIR, Université Pierre et Marie Curie-Paris 6, CNRS UMR 7222 4 place Jussieu, F-75252, Paris Cedex 05, France [email protected] Jean-Baptiste Mouret ISIR, Université Pierre et Marie Curie-Paris 6, CNRS UMR 7222 4 place Jussieu, F-75252, Paris Cedex 05, France [email protected] ABSTRACT Learning in robotics typically involves choosing a simple goal (e.g. walking) and assessing the performance of each con- troller with regard to this task (e.g. walking speed). How- ever, learning advanced, input-driven controllers (e.g. walk- ing in each direction) requires testing each controller on a large sample of the possible input signals. This costly pro- cess makes difficult to learn useful low-level controllers in robotics. Here we introduce BR-Evolution, a new evolutionary learn- ing technique that generates a behavioral repertoire by tak- ing advantage of the candidate solutions that are usually discarded. Instead of evolving a single, general controller, BR-evolution thus evolves a collection of simple controllers, one for each variant of the target behavior; to distinguish similar controllers, it uses a performance objective that al- lows it to produce a collection of diverse but high-performing behaviors. We evaluated this new technique by evolving gait controllers for a simulated hexapod robot. Results show that a single run of the EA quickly finds a collection of controllers that allows the robot to reach each point of the reachable space. Overall, BR-Evolution opens a new kind of learning algorithm that simultaneously optimizes all the achievable behaviors of a robot

Quality-diversity optimization: a novel branch of stochastic optimization

Traditional optimization algorithms search for a single global optimum that maximizes (or minimizes) the objective function. Multimodal optimization algorithms search for the highest peaks in the search space that can be more than one. Quality-Diversity algorithms are a recent addition to the evolutionary computation toolbox that do not only search for a single set of local optima, but instead try to illuminate the search space. In effect, they provide a holistic view of how high-performing solutions are distributed throughout a search space. The main differences with multimodal optimization algorithms are that (1) Quality-Diversity typically works in the behavioral space (or feature space), and not in the genotypic (or parameter) space, and (2) Quality-Diversity attempts to fill the whole behavior space, even if the niche is not a peak in the fitness landscape. In this chapter, we provide a gentle introduction to Quality-Diversity optimization, discuss the main representative algorithms, and the main current topics under consideration in the community. Throughout the chapter, we also discuss several successful applications of Quality-Diversity algorithms, including deep learning, robotics, and reinforcement learning

Quality-diversity optimization: a novel branch of stochastic optimization

Traditional optimization algorithms search for a single global optimum that maximizes (or minimizes) the objective function. Multimodal optimization algorithms search for the highest peaks in the search space that can be more than one. Quality-Diversity algorithms are a recent addition to the evolutionary computation toolbox that do not only search for a single set of local optima, but instead try to illuminate the search space. In effect, they provide a holistic view of how high-performing solutions are distributed throughout a search space. The main differences with multimodal optimization algorithms are that (1) Quality-Diversity typically works in the behavioral space (or feature space), and not in the genotypic (or parameter) space, and (2) Quality-Diversity attempts to fill the whole behavior space, even if the niche is not a peak in the fitness landscape. In this chapter, we provide a gentle introduction to Quality-Diversity optimization, discuss the main representative algorithms, and the main current topics under consideration in the community. Throughout the chapter, we also discuss several successful applications of Quality-Diversity algorithms, including deep learning, robotics, and reinforcement learning

The EUVE point of view of AD Leo

All the Extreme Ultraviolet Explorer (EUVE) observations of AD Leo, totalling 1.1 Ms of exposure time, have been employed to analyze the corona of this single M dwarf. The light curves show a well defined quiescent stage, and a distribution of amplitude of variability following a power law with a ~-2.4 index. The flaring behavior exhibits much similarity with other M active stars like FK Aqr or YY Gem, and flares behave differently from late type active giants and subgiants. The Emission Measure Distribution (EMD) of the summed spectrum, as well as that of quiescent and flaring stages, were obtained using a line-based method. The average EMD is dominated by material at log T(K)~6.9, with a second peak around log T(K)~6.3, and a large increase in the amount of material with log T(K)>~7.1 during flares, material almost absent during quiescence. The results are interpreted as the combination of three families of loops with maximum temperatures at log T(K)~6.3, ~6.9 and somewhere beyond log T(K)>~7.1. A value of the abundance of [Ne/Fe]=1.05+-0.08 was measured at log T(K)~5.9. No significative increment of Neon abundance was detected between quiescence and flaring states.Comment: Full PS version can be found also at http://www.astropa.unipa.it/~jsanz/papers0002.htm

Employing external facilitation to implement cognitive behavioral therapy in VA clinics: a pilot study

<p>Abstract</p> <p>Background</p> <p>Although for more than a decade healthcare systems have attempted to provide evidence-based mental health treatments, the availability and use of psychotherapies remains low. A significant need exists to identify simple but effective implementation strategies to adopt complex practices within complex systems of care. Emerging evidence suggests that facilitation may be an effective integrative implementation strategy for adoption of complex practices. The current pilot examined the use of external facilitation for adoption of cognitive behavioral therapy (CBT) in 20 Department of Veteran Affairs (VA) clinics.</p> <p>Methods</p> <p>The 20 clinics were paired on facility characteristics, and 23 clinicians from these were trained in CBT. A clinic in each pair was randomly selected to receive external facilitation. Quantitative methods were used to examine the extent of CBT implementation in 10 clinics that received external facilitation compared with 10 clinics that did not, and to better understand the relationship between individual providers' characteristics and attitudes and their CBT use. Costs of external facilitation were assessed by tracking the time spent by the facilitator and therapists in activities related to implementing CBT. Qualitative methods were used to explore contextual and other factors thought to influence implementation.</p> <p>Results</p> <p>Examination of change scores showed that facilitated therapists averaged an increase of 19% [95% CI: (2, 36)] in self-reported CBT use from baseline, while control therapists averaged a 4% [95% CI: (-14, 21)] increase. Therapists in the facilitated condition who were not providing CBT at baseline showed the greatest increase (35%) compared to a control therapist who was not providing CBT at baseline (10%) or to therapists in either condition who were providing CBT at baseline (average 3%). Increased CBT use was unrelated to prior CBT training. Barriers to CBT implementation were therapists' lack of control over their clinic schedule and poor communication with clinical leaders.</p> <p>Conclusions</p> <p>These findings suggest that facilitation may help clinicians make complex practice changes such as implementing an evidence-based psychotherapy. Furthermore, the substantial increase in CBT usage among the facilitation group was achieved at a modest cost.</p

Evolving Robots on Easy Mode: Towards a Variable Complexity Controller for Quadrupeds

The complexity of a legged robot's environment or task can inform how specialised its gait must be to ensure success. Evolving specialised robotic gaits demands many evaluations - acceptable for computer simulations, but not for physical robots. For some tasks, a more general gait, with lower optimization costs, could be satisfactory. In this paper, we introduce a new type of gait controller where complexity can be set by a single parameter, using a dynamic genotype-phenotype mapping. Low controller complexity leads to conservative gaits, while higher complexity allows more sophistication and high performance for demanding tasks, at the cost of optimization effort. We investigate the new controller on a virtual robot in simulations and do preliminary testing on a real-world robot. We show that having variable complexity allows us to adapt to different optimization budgets. With a high evaluation budget in simulation, a complex controller performs best. Moreover, real-world evolution with a limited evaluation budget indicates that a lower gait complexity is preferable for a relatively simple environment.Comment: Accepted to EvoApplications1

Driving of Outer Belt Electron Loss by Solar Wind Dynamic Pressure Structures : Analysis of Balloon and Satellite Data

We present observations of similar to 10-60 min solar wind dynamic pressure structures that drive large-scale coherent similar to 20-100 keV electron loss from the outer radiation belt. A combination of simultaneous satellite and Balloon Array for Radiation-belt Relativistic Electron Losses (BARREL) observations on 11-12 January 2014 shows a close association between the pressure structures and precipitation as inferred from BARREL X-rays. Specifically, the structures drive radial ExB transport of electrons up to 1 Earth radii, modulating the free electron energy available for low-frequency plasmaspheric hiss growth, and subsequent hiss-induced loss cone scattering. The dynamic pressure structures, originating near the Sun and commonly observed advecting with the solar wind, are thus able to switch on scattering loss of electrons by hiss over a large spatial scale. Our results provide a direct link between solar wind pressure fluctuations and modulation of electron loss from the outer radiation belt and may explain long-period modulations and large-scale coherence of X-rays commonly observed in the BARREL data set. Plain Language Summary The Earth's low-density magnetosphere is a region of enclosed magnetic field lines that contains energetic electrons ranging from eV to MeV energies. These populations can be greatly enhanced in response to solar driving. Following enhancements, energetic electron populations are depleted on timescales of hours to days by various processes. One important depletion process occurs when an electromagnetic plasma wave called plasmaspheric hiss, which exists within a high plasma density region called the plasmasphere and its (occasional) radial extension called the plume, scatters energetic electrons into the atmosphere. In this paper, we show that these hiss waves can be switched on by compressions of the magnetosphere which occur in response to similar to 1 hr long pressure structures in the solar wind. These structures originate at or near the Sun and are very common in the solar wind at 1 AU. The newly excited hiss waves scatter electrons into the atmosphere where they are observed on balloon-borne X-ray detectors. Our results suggest that magnetospheric models that predict the loss of electrons from hiss waves may be improved by consideration of solar wind pressure-driven dynamics.Peer reviewe

Combining two-directional synthesis and tandem reactions. Part 21: Exploitation of a dimeric macrocycle for chain terminus differentiation and synthesis of an sp3-rich library

The application of a tandem condensation/cyclisation/[3+2]-cycloaddition/elimination reaction gives an sp3-rich tricyclic pyrazoline scaffold with two ethyl esters in a single step from a simple linear starting material. The successive hydrolysis and cyclisation (with Boc anhydride) of these 3-dimensional architectures, generates unprecedented 16-membered macrocyclic bisanhydrides (characterised by XRD). Selective amidations could then be achieved by ring opening with a primary amine followed by HATU-promoted amide coupling to yield an sp3-rich natural product-like library