Importance of Parameter Settings on the Benefits of Robot-to-Robot Learning in Evolutionary Robotics

Robot-to-robot learning, a specific case of social learning in robotics, enables multiple robots to share learned skills while completing a task. The literature offers various statements of its benefits. Robots using this type of social learning can reach a higher performance, an increased learning speed, or both, compared to robots using individual learning only. No general explanation has been advanced for the difference in observations, which make the results highly dependent on the particular system and parameter setting. In this paper, we perform a detailed analysis into the effects of robot-to-robot learning. As a result, we show that this type of social learning can reduce the sensitivity of the learning process to the choice of parameters in two ways. First, robot-to-robot learning can reduce the number of bad performing individuals in the population. Second, robot-to-robot learning can increase the chance of having a successful run, where success is defined as the presence of a high performing individual. Additionally, we show that robot-to-robot learning results in an increased learning speed for almost all parameter settings. Our results indicate that robot-to-robot learning is a powerful mechanism which leads to benefits in both performance and learning speed

Tutorials at PPSN 2016

PPSN 2016 hosts a total number of 16 tutorials covering a broad range of current research in evolutionary computation. The tutorials range from introductory to advanced and specialized but can all be attended without prior requirements. All PPSN attendees are cordially invited to take this opportunity to learn about ongoing research activities in our field

Quantifying selection pressure

Selection is an essential component of any evolutionary system and analysing this fundamental force in evolution can provide relevant insights into the evolutionary development of a population. The 1990s and early 2000s saw a substantial number of publications that investigated selection pressure through methods such as takeover time and Markov chain analysis. Over the last decade, however, interest in the analysis of selection in evolutionary computing has waned. The established methods for analysis of selection pressure provide little insight when selection is based on more than comparison-of-fitness values. This can, for instance, be the case in coevolutionary systems, when measures unrelated to fitness affect the selection process (e.g., niching) or in systems that lack a crisply defined objective function. This article proposes two metrics that holistically consider the statistics of the evolutionary process to quantify selection pressure in evolutionary systems and so can be applied where traditionally used methods fall short. Themetrics are based on a statistical analysis of the relation between reproductive success and a quantifiable trait: one method builds on an estimate of the probability that this relation is random; the other uses a correlation measure. These metrics provide convenient tools to analyse selection pressure and so allow researchers to better understand this crucial component of evolutionary systems. Both metrics are straightforward to implement and can be used in post-hoc analyses as well as during the evolutionary process, for example, to inform parameter control mechanisms. Anumber of case studies and a critical analysis show that the proposed metrics provide relevant and reliable measures of selection pressure

Quadro generale e prospettive di cooperazione allo sviluppo

La presente bozza di ricerca, a cura del CONICS (Consorzio Interuniv. per la Cooperazione allo Sviluppo), delinea il quadro della situazione somala del tempo (1991) per quanto riguarda la storia, l'economia del paese, la cooperazione italiana allo sviluppo, il sistema educativo somalo e l'Università Nazionale Somala.Qabyaqoraalkan cilmibaariseed wuxuu ka hadlayaa xaaladda soomaaliyeed ee xilliga 1991 xagga taariikhda iyo dhaqaalaha dalka, habka waxbarashada soomaaliyeed iyo Jaamacadda Ummadda Soomaaliyeed.This research paper, edited by CONICS (Interuniversity Union for Development Cooperation), outlines the Somali situation up to 1991, concerning the history and the economy of the country, the italian cooperation for the development, the Somali educational system and the Somali National University

Towards empathic deep q-learning

As reinforcement learning (RL) scales to solve increasingly complex tasks, interest continues to grow in the fields of AI safety and machine ethics. As a contribution to these fields, this paper introduces an extension to Deep Q-Networks (DQNs), called Empathic DQN, that is loosely inspired both by empathy and the golden rule ("Do unto others as you would have them do unto you"). Empathic DQN aims to help mitigate negative side effects to other agents resulting from myopic goal-directed behavior. We assume a setting where a learning agent coexists with other independent agents (who receive unknown rewards), where some types of reward (e.g. negative rewards from physical harm) may generalize across agents. Empathic DQN combines the typical (self-centered) value with the estimated value of other agents, by imagining (by its own standards) the value of it being in the other's situation (by considering constructed states where both agents are swapped). Proof-of-concept results in two gridworld environments highlight the approach's potential to decrease collateral harms. While extending Empathic DQN to complex environments is non-trivial, we believe that this first step highlights the potential of bridge-work between machine ethics and RL to contribute useful priors for norm-abiding RL agents.Comment: To be presented as a poster at the IJCAI-19 AI Safety Worksho

Is Social Learning More Than Parameter Tuning?

Social learning enables multiple robots to share learned experiences while completing a task. The literature offers examples where robots trained with social learning reach a higher performance compared to their individual learning counterparts. No explanation has been advanced for that observation. In this research, we present experimental results suggesting that a lack of tuning of the parameters in social learning experiments could be the cause. In other words: the better the parameter settings are tuned, the less social learning can improve the system performance

Benefits of Social Learning in Physical Robots

Robot-to-robot learning, a specific case of social learning in robotics, enables the ability to transfer robot controllers directly from one robot to another. Previous studies showed that the exchange of controller information can increase learning speed and performance. However, most of these studies have been performed in simulation, where robots are identical. Therefore, the results do not necessarily transfer to a real environment, where each robot is unique per definition due to the random differences in hardware. In this paper, we investigate the effect of exchanging controller information, on top of individual learning, in a group of Thymio II robots for two tasks: obstacle avoidance and foraging. The controllers of the robots are neural networks that evolve using a modified version of the state-of-the-art NEAT algorithm, called cNEAT, which allows the conversion of innovations numbers from other robots. This paper shows that robot-to-robot learning seems to at least parallelise the search, reducing wall clock time. Additionally, controllers are less complex, resulting in a smaller search space

Towards empathic deep q-learning

As reinforcement learning (RL) scales to solve increasingly complex tasks, interest continues to grow in the fields of AI safety and machine ethics. As a contribution to these fields, this paper introduces an extension to Deep Q-Networks (DQNs), called Empathic DQN, that is loosely inspired both by empathy and the golden rule ("Do unto others as you would have them do unto you"). Empathic DQN aims to help mitigate negative side effects to other agents resulting from myopic goal-directed behavior. We assume a setting where a learning agent coexists with other independent agents (who receive unknown rewards), where some types of reward (e.g. negative rewards from physical harm) may generalize across agents. Empathic DQN combines the typical (self-centered) value with the estimated value of other agents, by imagining (by its own standards) the value of it being in the other's situation (by considering constructed states where both agents are swapped). Proof-of-concept results in two gridworld environments highlight the approach's potential to decrease collateral harms. While extending Empathic DQN to complex environments is non-trivial, we believe that this first step highlights the potential of bridge-work between machine ethics and RL to contribute useful priors for norm-abiding RL agents