Towards sample-efficient policy learning with DAC-ML

The sample-inefficiency problem in Artificial Intelligence refers to the inability of current Deep Reinforcement Learning models to optimize action policies within a small number of episodes. Recent studies have tried to overcome this limitation by adding memory systems and architectural biases to improve learning speed, such as in Episodic Reinforcement Learning. However, despite achieving incremental improvements, their performance is still not comparable to how humans learn behavioral policies. In this paper, we capitalize on the design principles of the Distributed Adaptive Control (DAC) theory of mind and brain to build a novel cognitive architecture (DAC-ML) that, by incorporating a hippocampus-inspired sequential memory system, can rapidly converge to effective action policies that maximize reward acquisition in a challenging foraging task

The EASEL project: Towards educational human-robot symbiotic interaction

This paper presents the EU EASEL project, which explores the potential impact and relevance of a robot in educational settings. We present the project objectives and the theorectical background on which the project builds, briefly introduce the EASEL technological developments, and end with a summary of what we have learned from the evaluation studies carried out in the project so far

Towards a synthetic tutor assistant: The EASEL project and its architecture

Robots are gradually but steadily being introduced in our daily lives. A paramount application is that of education, where robots can assume the role of a tutor, a peer or simply a tool to help learners in a specific knowledge domain. Such endeavor posits specific challenges: affective social behavior, proper modelling of the learner’s progress, discrimination of the learner’s utterances, expressions and mental states, which, in turn, require an integrated architecture combining perception, cognition and action. In this paper we present an attempt to improve the current state of robots in the educational domain by introducing the EASEL EU project. Specifically, we introduce the EASEL’s unified robot architecture, an innovative Synthetic Tutor Assistant (STA) whose goal is to interactively guide learners in a science-based learning paradigm, allowing us to achieve such rich multimodal interactions

Learning from a robot: creating synthetic psychologically plausible agents

Due to technological advancements, robots will soon become part of our daily lives and interact with us on a frequent basis. Robot acceptance is important, as it delineates whether users will potentially interact with them or not. We argue that psychological plausibility is a key determinant of acceptance and the challenge that rises is to understand, measure and identify what a ects plausibility. Here, we propose a taxonomy of four psychological benchmarks that one can apply to evaluate the behavioural components of robots and assess how they a ect acceptance: social competence, task competence, autonomy and morphology. By decomposing plausibility to discrete parts and empirically test them, we can use their interactions in practice for the meaningful design and development of social robots. In this thesis, we have identi ed behavioural components that are relevant to the proposed taxonomy and evaluated them in a series of studies. We show that it is possible to use the proposed taxonomy to evaluate the interaction and the robot. By systematically assessing the behavioural features of the robot, we gain useful insights that we apply to our H5WRobot that we later validate in the domain of tutoring. We show that our robot is accepted by students and stress that our proposed taxonomy might provide useful insights regarding the establishment of future assessments for HRI.A causa dels avenços tecnològics, els robots aviat formaran part de la nostra vida diària i interactuaran amb nosaltres de forma freqüent. Que els robots siguin ben rebuts és important, ja que determina si els usuaris voldran interactuar amb ells o no. Argumentem que la plausibilitat psicològica dels robots és fonamental per a la seva acceptació i que un repte que sorgeix és entendre, mesurar i identi car qué afecta aquesta plausibilitat. Proposem una taxonomia de quatre criteris psicològics que es poden aplicar per tal d'avaluar els components de conducta dels robots i com afecten la seva acceptació: competència social, competència funcional, autonomia i morfologia. Descomposant la plausibilitat en parts discretes, i avaluant-les de forma empírica, podem fer-ne un ús pràctic de les interaccions per al disseny i desenvolupament de robots socials. En aquesta tesi hem identi cat comportaments conductuals que són rellevants per a la taxonomia proposada i que han estat avaluats en una sèrie d'estudis. Mostrem que és possible utilitzar la taxonomia proposada per tal d'avaluar un robot i la interacció amb aquest. Mitjançant una avaluació sistemàtica de les caracterítiques conductuals dels robots, obtenim una sèrie d'idees útils que hem aplicat al nostre robot H5WRobot, i que posteriorment validem en un context de tutoria. Demostrem que el nostre robot és acceptat pels estudiants i fem palès que la taxonomia que proposem pot proporcionar observacions útils per a l'establiment de futures avaluacions per a la interacció entre humans i robots

Editorial: Living machines: from biological models to soft machines

Transferring the principles of living nature into living machines has preoccupied philosophers and scientists for more than 2000 years and inspired great minds like Leonardo Da Vinci to deliver outstanding inventions. In the last century, the study of nature and living organisms has led to innovations inspired by nature in a variety of different fields, such as engineer ing, architecture, materials sciences, medical technol ogy and robotics. The most prominent examples for commonly used bioinspired developments are Velcro (fastening inspired by burs hooks) and self-cleaning paints and coatings based on the principles of lotus plant leave

Object Localisation with a Highly Compliant Tactile Sensory Probe via Distributed Strain Sensors

Schultz M, Dürr V. Object Localisation with a Highly Compliant Tactile Sensory Probe via Distributed Strain Sensors. In: Vasiliki V, ed. Biomimetic and Biohybrid Systems. Lecture Notes in Computer Science. Vol 10928. Cham: Springer; 2018: 428-438.Insect antennae have been repeatedly proposed as paragons of active tactile sensors for biomimetic robots. A challenging aspect of using insect-like feelers for tactile localisation concerns the compliance of the long and slender structure of insect antennae. Other than in a rigid sensory probe, where the contact location in space may be estimated from the pointing direction and contact distance along the probe (polar coordinates), the strong compliance of insect antennae during contact events raises the question how insects can localise contact positions in space. Here we study the stick insect antenna to address this question. Our main objective was to test whether and how the bending properties of the insect antenna may allow reliable estimation of spatial contact locations through an array of bending sensors. During walking and climbing, the stick insect Carausius morosus executes cyclic antennal movements to explore the ambient space ahead. When the antenna touches an obstacle, it often bends strongly. Nevertheless, the insect can reliably reach for the contacted obstacle. Here, we systematically deflected insect antennae with an industrial robot to mimic an array of static contact locations. Then, we measured the resulting curvature of the flagellum, assuming that campanifom sensilla distributed along the flagellum could encode the corresponding bending profile. We found that we could train an artificial neural network to estimate the contact positions in 3D space with an accuracy of 0.5 mm or less from a given set of curvature data. This suggests that the bending characteristics of a tactile sensory probe could be tuned to aid spatial localisation by contact-sitedependent, compliant deformation

Estimating body pitch from distributed proprioception in a hexapod

Gollin A, Dürr V. Estimating body pitch from distributed proprioception in a hexapod. In: Vasiliki V, ed. Biomimetic and Biohybrid Systems. 7th International Conference, Living Machines 2018, Paris, France, July 17–20, 2018, Proceedings . Lecture Notes in Computer Science. Vol 10928. Cham: Springer; 2018: 187-199.Adaptability of legged locomotion relies on distributed proprioceptive feedback from the legs. Apart from low-level control of leg movements, proprioceptive cues may also be integrated to estimate overall locomotion parameters relevant to high-level control of behavior. For example, this couldbe relevant for reliable estimates of body inclination relative to the substrate, particularly in animals that lack dedicated graviceptors such as statocycsts. With regard to robotic systems, distributed proprioception could exploit physical interaction with the substrate to improve the robustness of inclinationestimates. In insect locomotion, it is unknown how overall parameters such as body inclination or forward velocity may be represented in the nervous system. If proprioceptive encoding was optimal, the afferent activity pattern of distributed proprioceptive cues from across the body should be a suitablerepresentation in itself. However, given noisy encoding in multiple afferent spike trains, it is unknown (i) how reliable the parameter estimates can be, and (ii) which parts of a distributed proprioceptive code are most relevant. Here we use a database on unrestrained whole-body kinematics of walkingand climbing stick insects in conjunction with simple spiking proprioceptor models to transform sets of joint angle time courses into corresponding sets of spike trains. In total, we tested four different types of models: a reference model without proprioceptive encoding and three proprioception models with differentfilter properties and spike generators. Within each model, we compared 4x4 conditions that differed in number and combination of joints and legs. Our results show that the contribution of middle and hind legs is of similar relevance for the estimation of body pitch, whereas front legs contribute only very little.Furthermore, femoral levation proved to be the most relevant degree of freedom, whereas estimates based on protraction and extension angles were less accurate

Complementary interactions between classical and top-down driven inhibitory mechanisms of attention

Selective attention informs decision-making by biasing perceptual processing towards task-relevant stimuli. In experimental and computational literature, this is most often implemented through top-down excitation of selected stimuli. However, physiological and anatomical evidence shows that in certain situations, top-down signals could instead be inhibitory. In this study, we investigated how such an inhibitory mechanism of top-down attention compares with an excitatory one. We did so in a neurorobotics context where the agent was controlled using an established hierarchical architecture. We augmented the architecture with an attentional system that implemented top-down attention biasing as connection gains. We tested four models of top-down attention on the simulated agent performing a foraging task: without top-down biasing, with only excitatory top-down gain, with only inhibitory top-down gain, and with both excitatory and inhibitory top-down gain. We manipulated the reward-distractor ratio that was presented and assessed the agent's performance using accumulated rewards and the latency of the selection. Using these measures, we provide evidence that excitatory and inhibitory mechanisms of attention complement each other.This project was funded by H2020 Research and Innovation program (#787061, ANITA), ERC H2020 (#840052, Cognitive RGS), H2020-Research and Innovation action EU.3.1.5.3 (#826421, VBC), and H2020-EU.2.1.5.1. (#820742, HR-Recycler). There are no conflicts of interest to declare

EFAA: a companion emerges from integrating a layered cognitive architecture

Presentat a: the 2014 ACM/IEEE International Conference on Human-Robot Interaction (HRI 2014), celebrat del 3 al 6 de març de 2014 a Bielefeld, Alemanya.In this video, we present the human robot interaction generated by applying the DAC cognitive architecture [1] on the iCub robot. We demonstrate how the robot reacts and adapts to its environment within the context a continuous interactive scenario including different games. We emphasize as well that the artificial agent is maintaining a self-model in terms of emotions and drives and how those are expressed in order affect the social interaction.This work is supported by the EU FP7 project EFAA (FP7-ICT- 270490) and by the Spanish Plan Nacional TIN2010-16745 (FAA-Arquitectura Cognitiva Biomimetica para un Funcional Ayudante de Androide Socialmente en Activo)