An approach to evolve and exploit repertoires of general robot behaviours

Recent works in evolutionary robotics have shown the viability of evolution driven by behavioural novelty and diversity. These evolutionary approaches have been successfully used to generate repertoires of diverse and high-quality behaviours, instead of driving evolution towards a single, task-specific solution. Having repertoires of behaviours can enable new forms of robotic control, in which high-level controllers continually decide which behaviour to execute. To date, however, only the use of repertoires of open-loop locomotion primitives has been studied. We propose EvoRBC-II, an approach that enables the evolution of repertoires composed of general closed-loop behaviours, that can respond to the robot's sensory inputs. The evolved repertoire is then used as a basis to evolve a transparent higher-level controller that decides when and which behaviours of the repertoire to execute. Relying on experiments in a simulated domain, we show that the evolved repertoires are composed of highly diverse and useful behaviours. The same repertoire contains sufficiently diverse behaviours to solve a wide range of tasks, and the EvoRBC-II approach can yield a performance that is comparable to the standard tabula-rasa evolution. EvoRBC-II enables automatic generation of hierarchical control through a two-step evolutionary process, thus opening doors for the further exploration of the advantages that can be brought by hierarchical control.info:eu-repo/semantics/acceptedVersio

A pilot study on the evolution of reward signals for hierarchical reinforcement learning

Recent research has shown that reinforcement learning agents can by greatly advantaged of the possibility of learning to select macro actions instead, or beside, fine primitive actions. The route usually followed to exploit this idea is to build agents with hierarchical architectures that can learn both a repertoire of macro actions and a macro policy that selects them, on the basis of the "final" reward signals related to the tasks to solve. This research presents a pre-liminary investigation that follows a different idea: evolving reinforcement signals that allow an agent to learn a repertoire of macro actions in an initial phase of life, and then to assemble and use them to solve different tasks, belonging to the same class, in a succeeding phase of life. The idea is preliminary tested with a simulated robot that has to search targets following coloured trails in a square arena. Results show that the idea is viable and that the emerged reinforcement signals allow the robot to develop macro actions that are actually useful to solve different tasks in succeeding phase of life

The morphofunctional approach to emotion modelling in robotics

In this conceptual paper, we discuss two areas of research in robotics, robotic models of emotion and morphofunctional machines, and we explore the scope for potential cross-fertilization between them. We shift the focus in robot models of emotion from information-theoretic aspects of appraisal to the interactive significance of bodily dispositions. Typical emotional phenomena such as arousal and action readiness can be interpreted as morphofunctional processes, and their functionality may be replicated in robotic systems with morphologies that can be modulated for real-time adaptation. We investigate the control requirements for such systems, and present a possible bio-inspired architecture, based on the division of control between neural and endocrine systems in humans and animals. We suggest that emotional epi- sodes can be understood as emergent from the coordination of action control and action-readiness, respectively. This stress on morphology complements existing research on the information-theoretic aspects of emotion

Modular and hierarchical brain organization to understand assimilation, accommodation and their relation to autism in reaching tasks: a developmental robotics hypothesis

By "assimilation" the child embodies the sensorimotor experience into already built mental structures. Conversely, by "accommodation" these structures are changed according to the child\u27s new experiences. Despite the intuitive power of these concepts to trace the course of sensorimotor development, they have gradually lost ground in psychology. This likely for a lack of brain related views capturing the dynamic mechanisms underlying them. Here we propose that brain modular and hierarchical organization is crucial to understanding assimilation/accommodation. We devised an experiment where a bio-inspired modular and hierarchical mixture-of-experts model guides a simulated robot to learn by trial-and-error different reaching tasks. The model gives a novel interpretation of assimilation/accommodation based on the functional organization of the experts allocated through learning. Assimilation occurs when the model adapts a copy of the expert trained for solving a task to face another task requiring similar sensorimotor mappings. Experts storing similar sensorimotor mappings belong to the same functional module. Accommodation occurs when the model uses non-trained experts to face tasks requiring different sensorimotor mappings (generating a new functional group of experts). The model provides a new theoretical framework to investigate impairments in assimilation/accommodation the autistic syndrome

On Simple Reactive Neural Networks for Behaviour-Based Reinforcement Learning

We present a behaviour-based reinforcement learning approach, inspired by Brook’s subsumption architecture, in which simple fully connected networks are trained as reactive behaviours. Our working assumption is that a pick and place robotic task can be simplified by leveraging domain knowledge of a robotics developer to decompose and train reactive behaviours; namely, approach, grasp, and retract. Then the robot autonomously learns how to combine reactive behaviours via an Actor-Critic architecture. We use an Actor-Critic policy to determine the activation and inhibition mechanisms of the reactive behaviours in a particular temporal sequence. We validate our approach in a simulated robot environment where the task is about picking a block and taking it to a target position while orienting the gripper from a top grasp. The latter represents an extra degree-of-freedom of which current end-to-end reinforcement learning approaches fail to generalise. Our findings suggest that robotic learning can be more effective if each behaviour is learnt in isolation and then combined them to accomplish the task. That is, our approach learns the pick and place task in 8,000 episodes, which represents a drastic reduction in the number of training episodes required by an end-to-end approach ( 95,000 episodes) and existing state-of-the-art algorithms

Evolving the behavior of machines: from micro to macroevolution

International audienceEvolution gave rise to creatures that are arguably more sophisticated than the greatest human-designed systems. This feat has inspired computer scientists since the advent of computing and led to optimization tools that can evolve complex neural networks for machines-an approach known as "neuroevolution". After a few successes in designing evolvable representations for high-dimensional artifacts, the field has been recently revitalized by going beyond optimization: to many, the wonder of evolution is less in the perfect optimization of each species than in the creativity of such a simple iterative process, that is, in the diversity of species. This modern view of artificial evolution is moving the field away from microevolution, following a fitness gradient in a niche, to macroevolution, filling many niches with highly different species. It already opened promising applications, like evolving gait repertoires, video game levels for different tastes, and diverse designs for aerodynamic bikes

Language impairment and colour categories

Goldstein (1948) reported multiple cases of failure to categorise colours in patients that he termed amnesic or anomic aphasics. these patients have a particular difficulty in producing perceptual categories in the absence of other aphasic impairments. we hold that neuropsychological evidence supports the view that the task of colour categorisation is logically impossible without labels