108 research outputs found
Final report key contents: main results accomplished by the EU-Funded project IM-CLeVeR - Intrinsically Motivated Cumulative Learning Versatile Robots
This document has the goal of presenting the main scientific and technological achievements of the project IM-CLeVeR. The document is organised as follows: 1. Project executive summary: a brief overview of the project vision, objectives and keywords. 2. Beneficiaries of the project and contacts: list of Teams (partners) of the project, Team Leaders and contacts. 3. Project context and objectives: the vision of the project and its overall objectives 4. Overview of work performed and main results achieved: a one page overview of the main results of the project 5. Overview of main results per partner: a bullet-point list of main results per partners 6. Main achievements in detail, per partner: a throughout explanation of the main results per partner (but including collaboration work), with also reference to the main publications supporting them
Learning Behaviors by an Autonomous Social Robot with Motivations
In this study, an autonomous social robot is living in a laboratory where it can interact with several items (people included). Its goal is to learn by itself the proper behaviors in order to maintain its well-being at as high a quality as possible. Several experiments have been conducted to test the performance of the system. The Object Q-Learning algorithm has been implemented in the robot as the learning algorithm. This algorithm is a variation of the traditional Q-Learning because it considers a reduced state space and collateral effects. The comparison of the performance of both algorithms is shown in the first part of the experiments. Moreover, two mechanisms intended to reduce the learning session durations have been included: Well-Balanced Exploration and Amplified Reward. Their advantages are justified in the results obtained in the second part of the experiments. Finally, the behaviors learned by our robot are analyzed. The resulting behaviors have not been preprogrammed. In fact, they have been learned by real interaction in the real world and are related to the motivations of the robot. These are natural behaviors in the sense that they can be easily understood by humans observing the robot.The authors gratefully acknowledge the funds provided by the Spanish Government through the project call "Aplicaciones de los robots sociales", DPI2011-26980 from the Spanish Ministry of Economy and Competitiveness.Publicad
Lâauto-exploration des espaces sensorimoteurs chez les robots
Developmental robotics has begun in the last fifteen years to study robots that havea childhoodâcrawling before trying to run, playing before being usefulâand that are basing their decisions upon a lifelong and embodied experience of the real-world. In this context, this thesis studies sensorimotor explorationâthe discovery of a robotâs own body and proximal environmentâduring the early developmental stages, when no prior experience of the world is available. Specifically, we investigate how to generate a diversity of effects in an unknown environment. This approach distinguishes itself by its lack of user-defined reward or fitness function, making it especially suited for integration in self-sufficient platforms. In a first part, we motivate our approach, formalize the exploration problem, define quantitative measures to assess performance, and propose an architectural framework to devise algorithms. through the extensive examination of a multi-joint arm example, we explore some of the fundamental challenges that sensorimotor exploration faces, such as high-dimensionality and sensorimotor redundancy, in particular through a comparison between motor and goal babbling exploration strategies. We propose several algorithms and empirically study their behaviour, investigating the interactions with developmental constraints, external demonstrations and biologicallyinspired motor synergies. Furthermore, because even efficient algorithms can provide disastrous performance when their learning abilities do not align with the environmentâs characteristics, we propose an architecture that can dynamically discriminate among a set of exploration strategies. Even with good algorithms, sensorimotor exploration is still an expensive propositionâ a problem since robots inherently face constraints on the amount of data they are able to gather; each observation takes a non-negligible time to collect. [...] Throughout this thesis, our core contributions are algorithms description and empirical results. In order to allow unrestricted examination and reproduction of all our results, the entire code is made available. Sensorimotor exploration is a fundamental developmental mechanism of biological systems. By decoupling it from learning and studying it in its own right in this thesis, we engage in an approach that casts light on important problems facing robots developing on their own.La robotique dĂ©veloppementale a entrepris, au courant des quinze derniĂšres annĂ©es,dâĂ©tudier les processus dĂ©veloppementaux, similaires Ă ceux des systĂšmes biologiques,chez les robots. Le but est de crĂ©er des robots qui ont une enfanceâqui rampent avant dâessayer de courir, qui jouent avant de travaillerâet qui basent leurs dĂ©cisions sur lâexpĂ©rience de toute une vie, incarnĂ©s dans le monde rĂ©el.Dans ce contexte, cette thĂšse Ă©tudie lâexploration sensorimotriceâla dĂ©couverte pour un robot de son propre corps et de son environnement procheâpendant les premiers stage du dĂ©veloppement, lorsque quâaucune expĂ©rience prĂ©alable du monde nâest disponible. Plus spĂ©cifiquement, cette thĂšse se penche sur comment gĂ©nĂ©rer une diversitĂ© dâeffets dans un environnement inconnu. Cette approche se distingue par son absence de fonction de rĂ©compense ou de fitness dĂ©finie par un expert, la rendant particuliĂšrement apte Ă ĂȘtre intĂ©grĂ©e sur des robots auto-suffisants.Dans une premiĂšre partie, lâapproche est motivĂ©e et le problĂšme de lâexploration est formalisĂ©, avec la dĂ©finition de mesures quantitatives pour Ă©valuer le comportement des algorithmes et dâun cadre architectural pour la crĂ©ation de ces derniers. Via lâexamen dĂ©taillĂ© de lâexemple dâun bras robot Ă multiple degrĂ©s de libertĂ©, la thĂšse explore quelques unes des problĂ©matiques fondamentales que lâexploration sensorimotrice pose, comme la haute dimensionnalitĂ© et la redondance sensorimotrice. Cela est fait en particulier via la comparaison entre deux stratĂ©gies dâexploration: le babillage moteur et le babillage dirigĂ© par les objectifs. Plusieurs algorithmes sont proposĂ©s tour Ă tour et leur comportement est Ă©valuĂ© empiriquement, Ă©tudiant les interactions qui naissent avec les contraintes dĂ©veloppementales, les dĂ©monstrations externes et les synergies motrices. De plus, parce que mĂȘme des algorithmes efficaces peuvent se rĂ©vĂ©ler terriblement inefficaces lorsque leurs capacitĂ©s dâapprentissage ne sont pas adaptĂ©s aux caractĂ©ristiques de leur environnement, une architecture est proposĂ©e qui peut dynamiquement choisir la stratĂ©gie dâexploration la plus adaptĂ©e parmi un ensemble de stratĂ©gies. Mais mĂȘme avec de bons algorithmes, lâexploration sensorimotrice reste une entreprise coĂ»teuseâun problĂšme important, Ă©tant donnĂ© que les robots font face Ă des contraintes fortes sur la quantitĂ© de donnĂ©es quâils peuvent extraire de leur environnement;chaque observation prenant un temps non-nĂ©gligeable Ă rĂ©cupĂ©rer. [...] Ă travers cette thĂšse, les contributions les plus importantes sont les descriptions algorithmiques et les rĂ©sultats expĂ©rimentaux. De maniĂšre Ă permettre la reproduction et la rĂ©examination sans contrainte de tous les rĂ©sultats, lâensemble du code est mis Ă disposition. Lâexploration sensorimotrice est un mĂ©canisme fondamental du dĂ©veloppement des systĂšmes biologiques. La sĂ©parer dĂ©libĂ©rĂ©ment des mĂ©canismes dâapprentissage et lâĂ©tudier pour elle-mĂȘme dans cette thĂšse permet dâĂ©clairer des problĂšmes importants que les robots se dĂ©veloppant seuls seront amenĂ©s Ă affronter
Computational visual attention systems and their cognitive foundation: A survey
Permission to make digital/hard copy of all or part of this material without fee for personal
or classroom use provided that the copies are not made or distributed for profit or commercial
advantage, the ACM copyright/server notice, the title of the publication, and its date appear, and
notice is given that copying is by permission of the ACM, Inc. To copy otherwise, to republish,
to post on servers, or to redistribute to lists requires prior specific permission and/or a fee.
(c) 2010 ACMBased on concepts of the human visual system, computational visual attention systems aim to
detect regions of interest in images. Psychologists, neurobiologists, and computer scientists have
investigated visual attention thoroughly during the last decades and profited considerably from
each other. However, the interdisciplinarity of the topic holds not only benefits but also difficulties:
concepts of other fields are usually hard to access due to differences in vocabulary and lack of
knowledge of the relevant literature. This paper aims to bridge this gap and bring together
concepts and ideas from the different research areas. It provides an extensive survey of the
grounding psychological and biological research on visual attention as well as the current state
of the art of computational systems. Furthermore, it presents a broad range of applications
of computational attention systems in fields like computer vision, cognitive systems and mobile
robotics. We conclude with a discussion on the limitations and open questions in the field
Computational visual attention systems and their cognitive foundation: A survey
Permission to make digital/hard copy of all or part of this material without fee for personal
or classroom use provided that the copies are not made or distributed for profit or commercial
advantage, the ACM copyright/server notice, the title of the publication, and its date appear, and
notice is given that copying is by permission of the ACM, Inc. To copy otherwise, to republish,
to post on servers, or to redistribute to lists requires prior specific permission and/or a fee.
(c) 2010 ACMBased on concepts of the human visual system, computational visual attention systems aim to
detect regions of interest in images. Psychologists, neurobiologists, and computer scientists have
investigated visual attention thoroughly during the last decades and profited considerably from
each other. However, the interdisciplinarity of the topic holds not only benefits but also difficulties:
concepts of other fields are usually hard to access due to differences in vocabulary and lack of
knowledge of the relevant literature. This paper aims to bridge this gap and bring together
concepts and ideas from the different research areas. It provides an extensive survey of the
grounding psychological and biological research on visual attention as well as the current state
of the art of computational systems. Furthermore, it presents a broad range of applications
of computational attention systems in fields like computer vision, cognitive systems and mobile
robotics. We conclude with a discussion on the limitations and open questions in the field
Autonomous Decision-Making based on Biological Adaptive Processes for Intelligent Social Robots
MenciĂłn Internacional en el tĂtulo de doctorThe unceasing development of autonomous robots in many different scenarios drives a
new revolution to improve our quality of life. Recent advances in human-robot interaction
and machine learning extend robots to social scenarios, where these systems pretend
to assist humans in diverse tasks. Thus, social robots are nowadays becoming real in
many applications like education, healthcare, entertainment, or assistance. Complex
environments demand that social robots present adaptive mechanisms to overcome
different situations and successfully execute their tasks. Thus, considering the previous
ideas, making autonomous and appropriate decisions is essential to exhibit reasonable
behaviour and operate well in dynamic scenarios.
Decision-making systems provide artificial agents with the capacity of making
decisions about how to behave depending on input information from the environment.
In the last decades, human decision-making has served researchers as an inspiration to
endow robots with similar deliberation. Especially in social robotics, where people expect
to interact with machines with human-like capabilities, biologically inspired decisionmaking
systems have demonstrated great potential and interest. Thereby, it is expected
that these systems will continue providing a solid biological background and improve the
naturalness of the human-robot interaction, usability, and the acceptance of social robots
in the following years.
This thesis presents a decision-making system for social robots acting in healthcare,
entertainment, and assistance with autonomous behaviour. The systemâs goal is to
provide robots with natural and fluid human-robot interaction during the realisation of
their tasks. The decision-making system integrates into an already existing software
architecture with different modules that manage human-robot interaction, perception,
or expressiveness. Inside this architecture, the decision-making system decides which
behaviour the robot has to execute after evaluating information received from different
modules in the architecture. These modules provide structured data about planned
activities, perceptions, and artificial biological processes that evolve with time that are the
basis for natural behaviour. The natural behaviour of the robot comes from the evolution
of biological variables that emulate biological processes occurring in humans. We also
propose a Motivational model, a module that emulates biological processes in humans for
generating an artificial physiological and psychological state that influences the robotâs
decision-making. These processes emulate the natural biological rhythms of the human organism to produce biologically inspired decisions that improve the naturalness exhibited
by the robot during human-robot interactions. The robotâs decisions also depend on what
the robot perceives from the environment, planned events listed in the robotâs agenda, and
the unique features of the user interacting with the robot.
The robotâs decisions depend on many internal and external factors that influence how
the robot behaves. Users are the most critical stimuli the robot perceives since they are
the cornerstone of interaction. Social robots have to focus on assisting people in their
daily tasks, considering that each person has different features and preferences. Thus,
a robot devised for social interaction has to adapt its decisions to people that aim at
interacting with it. The first step towards adapting to different users is identifying the user
it interacts with. Then, it has to gather as much information as possible and personalise
the interaction. The information about each user has to be actively updated if necessary
since outdated information may lead the user to refuse the robot. Considering these facts,
this work tackles the user adaptation in three different ways.
âą The robot incorporates user profiling methods to continuously gather information
from the user using direct and indirect feedback methods.
âą The robot has a Preference Learning System that predicts and adjusts the userâs
preferences to the robotâs activities during the interaction.
âą An Action-based Learning System grounded on Reinforcement Learning is
introduced as the origin of motivated behaviour.
The functionalities mentioned above define the inputs received by the decisionmaking
system for adapting its behaviour. Our decision-making system has been designed
for being integrated into different robotic platforms due to its flexibility and modularity.
Finally, we carried out several experiments to evaluate the architectureâs functionalities
during real human-robot interaction scenarios. In these experiments, we assessed:
âą How to endow social robots with adaptive affective mechanisms to overcome
interaction limitations.
âą Active user profiling using face recognition and human-robot interaction.
âą A Preference Learning System we designed to predict and adapt the user
preferences towards the robotâs entertainment activities for adapting the interaction.
âą A Behaviour-based Reinforcement Learning System that allows the robot to learn
the effects of its actions to behave appropriately in each situation.
âą The biologically inspired robot behaviour using emulated biological processes and
how the robot creates social bonds with each user.
âą The robotâs expressiveness in affect (emotion and mood) and autonomic functions
such as heart rate or blinking frequency.Programa de Doctorado en IngenierĂa ElĂ©ctrica, ElectrĂłnica y AutomĂĄtica por la Universidad Carlos III de MadridPresidente: Richard J. Duro FernĂĄndez.- Secretaria: ConcepciĂłn Alicia Monje Micharet.- Vocal: Silvia Ross
Goal Reasoning: Papers from the ACS Workshop
This technical report contains the 14 accepted papers presented at the Workshop on Goal Reasoning,
which was held as part of the 2015 Conference on Advances in Cognitive Systems (ACS-15) in Atlanta,
Georgia on 28 May 2015. This is the fourth in a series of workshops related to this topic, the first of
which was the AAAI-10 Workshop on Goal-Directed Autonomy; the second was the Self-Motivated
Agents (SeMoA) Workshop, held at Lehigh University in November 2012; and the third was the Goal
Reasoning Workshop at ACS-13 in Baltimore, Maryland in December 2013
Intrinsic Motivation in Computational Creativity Applied to Videogames
PhD thesisComputational creativity (CC) seeks to endow artificial systems with creativity.
Although human creativity is known to be substantially driven by
intrinsic motivation (IM), most CC systems are extrinsically motivated. This
restricts their actual and perceived creativity and autonomy, and consequently
their benefit to people. In this thesis, we demonstrate, via theoretical arguments
and through applications in videogame AI, that computational intrinsic
reward and models of IM can advance core CC goals.
We introduce a definition of IM to contextualise related work. Via two
systematic reviews, we develop typologies of the benefits and applications of
intrinsic reward and IM models in CC and game AI. Our reviews highlight
that related work is limited to few reward types and motivations, and we thus
investigate the usage of empowerment, a little studied, information-theoretic
intrinsic reward, in two novel models applied to game AI.
We define coupled empowerment maximisation (CEM), a social IM model,
to enable general co-creative agents that support or challenge their partner
through emergent behaviours. Via two qualitative, observational vignette
studies on a custom-made videogame, we explore CEMâs ability to drive
general and believable companion and adversary non-player characters which
respond creatively to changes in their abilities and the game world.
We moreover propose to leverage intrinsic reward to estimate peopleâs
experience of interactive artefacts in an autonomous fashion. We instantiate
this proposal in empowerment-based player experience prediction (EBPXP)
and apply it to videogame procedural content generation. By analysing think-aloud
data from an experiential vignette study on a dedicated game, we
identify several experiences that EBPXP could predict.
Our typologies serve as inspiration and reference for CC and game AI
researchers to harness the benefits of IM in their work. Our new models can
increase the generality, autonomy and creativity of next-generation videogame
AI, and of CC systems in other domains
Affective Computing
This book provides an overview of state of the art research in Affective Computing. It presents new ideas, original results and practical experiences in this increasingly important research field. The book consists of 23 chapters categorized into four sections. Since one of the most important means of human communication is facial expression, the first section of this book (Chapters 1 to 7) presents a research on synthesis and recognition of facial expressions. Given that we not only use the face but also body movements to express ourselves, in the second section (Chapters 8 to 11) we present a research on perception and generation of emotional expressions by using full-body motions. The third section of the book (Chapters 12 to 16) presents computational models on emotion, as well as findings from neuroscience research. In the last section of the book (Chapters 17 to 22) we present applications related to affective computing
- âŠ