Attentional mechanisms for socially interactive robots – a survey

This review intends to provide an overview of the state of the art in the modeling and implementation of automatic attentional mechanisms for socially interactive robots. Humans assess and exhibit intentionality by resorting to multisensory processes that are deeply rooted within low-level automatic attention-related mechanisms of the brain. For robots to engage with humans properly, they should also be equipped with similar capabilities. Joint attention, the precursor of many fundamental types of social interactions, has been an important focus of research in the past decade and a half, therefore providing the perfect backdrop for assessing the current status of state-of-the-art automatic attentional-based solutions. Consequently, we propose to review the influence of these mechanisms in the context of social interaction in cutting-edge research work on joint attention. This will be achieved by summarizing the contributions already made in these matters in robotic cognitive systems research, by identifying the main scientific issues to be addressed by these contributions and analyzing how successful they have been in this respect, and by consequently drawing conclusions that may suggest a roadmap for future successful research efforts

An end-to-end review of gaze estimation and its interactive applications on handheld mobile devices

In recent years we have witnessed an increasing number of interactive systems on handheld mobile devices which utilise gaze as a single or complementary interaction modality. This trend is driven by the enhanced computational power of these devices, higher resolution and capacity of their cameras, and improved gaze estimation accuracy obtained from advanced machine learning techniques, especially in deep learning. As the literature is fast progressing, there is a pressing need to review the state of the art, delineate the boundary, and identify the key research challenges and opportunities in gaze estimation and interaction. This paper aims to serve this purpose by presenting an end-to-end holistic view in this area, from gaze capturing sensors, to gaze estimation workflows, to deep learning techniques, and to gaze interactive applications.PostprintPeer reviewe

Memory and mental time travel in humans and social robots.

From neuroscience, brain imaging and the psychology of memory, we are beginning to assemble an integrated theory of the brain subsystems and pathways that allow the compression, storage and reconstruction of memories for past events and their use in contextualizing the present and reasoning about the future-mental time travel (MTT). Using computational models, embedded in humanoid robots, we are seeking to test the sufficiency of this theoretical account and to evaluate the usefulness of brain-inspired memory systems for social robots. In this contribution, we describe the use of machine learning techniques-Gaussian process latent variable models-to build a multimodal memory system for the iCub humanoid robot and summarize results of the deployment of this system for human-robot interaction. We also outline the further steps required to create a more complete robotic implementation of human-like autobiographical memory and MTT. We propose that generative memory models, such as those that form the core of our robot memory system, can provide a solution to the symbol grounding problem in embodied artificial intelligence. This article is part of the theme issue 'From social brains to social robots: applying neurocognitive insights to human-robot interaction'.Funding. The preparation of this chapter was supported by funding from the EU Seventh Framework Programme as part of the projects Experimental Functional Android Assistant (EFAA, FP7-ICT-270490) and What You Say Is What You Did (WYSIWYD, FP7-ICT-612139) and by the EU H2020 Programme as part of the Human Brain Project (HBP-SGA1, 720270; HBP-SGA2, 785907). Acknowledgements. The authors are grateful to Paul Verschure, Peter Dominey, Giorgio Metta, Yiannis Demiris and the other members of the WYSIWYD and EFAA consortia; to members of the HBP EPISENSE group; and to our colleagues at the University of Sheffield who have helped us to develop memory systems for the iCub, particularly Luke Boorman, Harry Jackson and Matthew Evans. The Sheffield iCub was purchased with the support of the UK Engineering and Physical Sciences Research Council (EPSRC)

Intelligent Interfaces to Empower People with Disabilities

Severe motion impairments can result from non-progressive disorders, such as cerebral palsy, or degenerative neurological diseases, such as Amyotrophic Lateral Sclerosis (ALS), Multiple Sclerosis (MS), or muscular dystrophy (MD). They can be due to traumatic brain injuries, for example, due to a traffic accident, or to brainste

Motion-based technology to support motor skills screening in developing children: A scoping review

Background. Acquiring motor skills is fundamental for children's development since it is linked to cognitive development. However, access to early detection of motor development delays is limited. Aim. This review explores the use and potential of motion-based technology (MBT) as a complement to support and increase access to motor screening in developing children. Methods. Six databases were searched following the PRISMA guidelines to search, select, and assess relevant works where MBT recognised the execution of children's motor skills. Results. 164 studies were analysed to understand the type of MBT used, the motor skills detected, the purpose of using MBT and the age group targeted. Conclusions. There is a gap in the literature aiming to integrate MBT in motor skills development screening and assessment processes. Depth sensors are the prevailing technology offering the largest detection range for children from age 2. Nonetheless, the motor skills detected by MBT represent about half of the motor skills usually observed to screen and assess motor development. Overall, research in this field is underexplored. The use of multimodal approaches, combining various motion-based sensors, may support professionals in the health domain and increase access to early detection programmes.Funding for open access charge: Universidad de Málaga / CBUA

Multisensory learning in adaptive interactive systems

The main purpose of my work is to investigate multisensory perceptual learning and sensory integration in the design and development of adaptive user interfaces for educational purposes. To this aim, starting from renewed understanding from neuroscience and cognitive science on multisensory perceptual learning and sensory integration, I developed a theoretical computational model for designing multimodal learning technologies that take into account these results. Main theoretical foundations of my research are multisensory perceptual learning theories and the research on sensory processing and integration, embodied cognition theories, computational models of non-verbal and emotion communication in full-body movement, and human-computer interaction models. Finally, a computational model was applied in two case studies, based on two EU ICT-H2020 Projects, "weDRAW" and "TELMI", on which I worked during the PhD