Online Weighted Q-Ensembles for Reduced Hyperparameter Tuning in Reinforcement Learning

Reinforcement learning is a promising paradigm for learning robot control, allowing complex control policies to be learned without requiring a dynamics model. However, even state of the art algorithms can be difficult to tune for optimum performance. We propose employing an ensemble of multiple reinforcement learning agents, each with a different set of hyperparameters, along with a mechanism for choosing the best performing set(s) on-line. In the literature, the ensemble technique is used to improve performance in general, but the current work specifically addresses decreasing the hyperparameter tuning effort. Furthermore, our approach targets on-line learning on a single robotic system, and does not require running multiple simulators in parallel. Although the idea is generic, the Deep Deterministic Policy Gradient was the model chosen, being a representative deep learning actor-critic method with good performance in continuous action settings but known high variance. We compare our online weighted q-ensemble approach to q-average ensemble strategies addressed in literature using alternate policy training, as well as online training, demonstrating the advantage of the new approach in eliminating hyperparameter tuning. The applicability to real-world systems was validated in common robotic benchmark environments: the bipedal robot half cheetah and the swimmer. Online Weighted Q-Ensemble presented overall lower variance and superior results when compared with q-average ensembles using randomized parameterizations

International student mobility and labour market outcomes: an investigation of the role of level of study, type of mobility, and international prestige hierarchies

Over the last decades, there has been increasing interest in the topic of international student mobility (ISM). However, there is surprisingly little analysis of the ways in which different characteristics and types of short-term ISM or the importance of host education systems and labour markets may affect early career outcomes of formerly mobile graduates. Therefore, in this study we explore, first, the relationship between participation in ISM at the Bachelor and Master level and graduates’ wages and the duration of education-to-work transitions. Second, we investigate variations in ISM’s labour market outcomes according to the type of mobility: study, internships, or combinations of both. Third, we examine the relationship between labour market outcomes of formerly mobile students and the country of destination’s position in higher education international prestige hierarchies and labour market competitiveness. We use the Dutch National Alumni Survey 2015, a representative survey of higher education graduates in the Netherlands, conducted 1.5 years after graduation. Before controlling for selection into ISM, the results suggest the existence of labour market returns to ISM and that the heterogeneity of ISM experiences matters, as labour market outcomes vary according to the level of study, the type of mobility and the positioning of the country of destination in international prestige hierarchies. However, after controlling for selection into ISM through propensity score matching, the differences in early career outcomes between formerly mobile and non-mobile graduates disappear, suggesting that they cannot be causally attributed to their ISM-experience. We explain these results with reference to the characteristics of the Dutch education system and labour market, where restricted possibilities for upward vertical mobility limit returns to ISM in the local labour market

Collision-free coordination of a group of unicycle mobile robots

Abstract only

Attraction of Trichogramma Wasps to Butterfly Oviposition-Induced Plant Volatiles Depends on Brassica Species, Wasp Strain and Leaf Necrosis

Within the Brassicaceae, wild as well as crop species are challenged by specialist herbivores including cabbage white butterflies (Pieris spp.). The wild crucifer Brassica nigra responds to oviposition by Pieris butterflies by the synergistic expression of two egg-killing traits. Genotypes that express a hypersensitive response (HR)-like necrosis (direct egg-killing) also emit oviposition-induced plant volatiles (OIPVs) attracting Trichogramma egg parasitoids (indirect egg-killing). This so-called double defense line can result in high butterfly egg mortalities. It remains unknown whether this strategy is unique to B. nigra or more common in Brassica species. To test this, we examined the response of different Trichogramma evanescens lines to OIPVs emitted by B. nigra and three close relatives (Brassica napus, Brassica rapa, and Brassica oleracea). Furthermore, we evaluated whether HR-like necrosis played a role in the attraction toward plant volatiles. Our results show a specificity in wasp attraction to different plant species. Three out of four plant species attracted a specific T. evanescens strain, including the crops B. rapa and B. napus. Parasitoid attraction was positively affected by presence of HR-like necrosis in one plant species. Our findings imply that, despite being a true generalist in terms of host range, T. evanescens shows intraspecific variation during host searching, which should be taken into account when selecting parasitoid lines for biocontrol of certain crops. Finally, we conclude that also crop plants within the Brassicaceae family possess egg-killing traits and can exert the double-defense line which may enable effective selection of egg-killing defense traits by cabbage breeders

Tech United Eindhoven RoboCup adult size humanoid team description 2012

This document presents the 2012 Tech United Eindhoven adult size humanoid robot team from The Netherlands. The team contributes the adult-size humanoid robot TUlip. Here we present the mechanical design and kinematic structure of the robot. We introduce the walking gait and contribute a controller structure including gravity compensation. Finally, we describe the vision system, self localization and world model, which are used for the attacker and defender strategy in the humanoid robot soccer game