Proactive learning of cognitive exercises with a social robot

Extended Abstract presentat al workshop de Prediction and Anticipation Reasoning in Human Robot Interaction (https://www.iri.upc.edu/workshops/pred-ant-hri/cfp.html), a la conferència ICRA 2022. Aquest article descriu un treball en progrés que es preveu afegir a la tesi. A dia d'aquesta descripció (15/07/2022), encara no s'ha publicat un enllaç al document presentat al workshop.We introduce INtuitive PROgramming 2 (INPRO2), an improvement over our previous INPRO framework for learning board exercises via demonstrations. INPRO2 makes use of our Online Action Recognition through Unification (OARU) algorithm, which maintains and extends as needed a library of STRIPS action schemata that represent the dynamics, rules and goal of the exercise. OARU operates on a sequence of states shown by the user. Each state transition is either used to learn a new action, or is recognized as an instance of one action currently present in the library, possibly refining it. We have extended OARU to support negative examples (i.e. invalid moves that show forbidden state transitions) in order to increase the complexity of the exercises that can be learned. This new OARU's feature is exploited through another crucial element of INPRO2: its ability to proactively ask for the legality of certain moves to the user in critical situations, and fix overly permissive actions. We show an example of a typical INPRO2 learning session. We also outline a plan for a user study that will serve to assess the proactive behavior of the robot.Preprin

Implications of robot backchannelling in cognitive therapy

The social ability of humans to provide active feedback during conversations is known as backchannelling. Recent work has recognised the importance of endowing robots with such social behaviour to make interactions more natural. Nonetheless, very little is known about how backchannelling should be designed in order to be detected and whether it can have an impact on users' behaviour and performance in cooperative tasks. In this article, we aim at evaluating the legibility of robot's backchannelling behaviour on Persons with Dementia (PwDs) and its effect on their performance when playing cognitive training exercises. Aiming to do so, a TIAGo robot was endowed with backchannelling behaviour generated by combining verbal and non-verbal cues. To evaluate our system, two user studies were carried out, in which the social signal was provided first by a human therapist and later on by a robot. Results indicate that patients were capable of identifying such kind of feedback. Nonetheless, our findings pointed out a significant difference in terms of performance between the two studies. They reveal how patients in the study with the robot overused the feedback to obtain the correct answer, putting in place a cheating mechanism that has led them to significantly worsen their performance. We conclude our work by discussing the implications of our findings when deploying robots in sensitive roles and possible solutions to address such unexpected behaviours.Peer ReviewedPostprint (author's final draft

Automatic learning of cognitive exercises for socially assistive robotics

© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksIn this paper, we present a learning approach to facilitate the teaching of new board exercises to assistive robotic systems. We formulate the problem as the learning of action models using Boolean predicates, disjunctive preconditions, and existential quantifiers from demonstrations of successful exercise executions. To be able to cope with exercises whose rules depend on a set of features that are initialized at the beginning of each play-out, we introduce the concept of dynamic context. Furthermore, we show how the learnt knowledge can be represented intuitively in a graphical interface that helps the caregiver understand what the system has learnt. As validation, we conducted a user study in which we evaluated whether and to which extent different types of feedback can affect the subjects’ performance while teaching three types of exercises: (1) sorting numbers; (2) arranging letters; and (3) reproducing shapes sequences in reversed order. The results suggest that textual and graphical feedback are beneficial.A. Andriella, C. Torras and A. Suarez-Hern ´ andez were partially funded ´ by the European Union´s Horizon 2020 under ERC Advanced Grant CLOTHILDE (no. 741930), G. Alenya by the EU H2020 research and ìnnovation programme IMAGINE (no. 731761) and J. Segovia-Aguas by the programme TAILOR (no. 952215). The work was partially supported by the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656)Peer ReviewedAward-winningPostprint (author's final draft

Automatic learning of cognitive exercises for socially assistive robotics

© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksIn this paper, we present a learning approach to facilitate the teaching of new board exercises to assistive robotic systems. We formulate the problem as the learning of action models using Boolean predicates, disjunctive preconditions, and existential quantifiers from demonstrations of successful exercise executions. To be able to cope with exercises whose rules depend on a set of features that are initialized at the beginning of each play-out, we introduce the concept of dynamic context. Furthermore, we show how the learnt knowledge can be represented intuitively in a graphical interface that helps the caregiver understand what the system has learnt. As validation, we conducted a user study in which we evaluated whether and to which extent different types of feedback can affect the subjects’ performance while teaching three types of exercises: (1) sorting numbers; (2) arranging letters; and (3) reproducing shapes sequences in reversed order. The results suggest that textual and graphical feedback are beneficial.A. Andriella, C. Torras and A. Suarez-Hern ´ andez were partially funded ´ by the European Union´s Horizon 2020 under ERC Advanced Grant CLOTHILDE (no. 741930), G. Alenya by the EU H2020 research and ìnnovation programme IMAGINE (no. 731761) and J. Segovia-Aguas by the programme TAILOR (no. 952215). The work was partially supported by the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656)Peer ReviewedAward-winningPostprint (author's final draft

Introducing CARESSER: A framework for in situ learning robot social assistance from expert knowledge and demonstrations

Socially assistive robots have the potential to augment and enhance therapist’s effectiveness in repetitive tasks such as cognitive therapies. However, their contribution has generally been limited as domain experts have not been fully involved in the entire pipeline of the design process as well as in the automatisation of the robots’ behaviour. In this article, we present aCtive leARning agEnt aSsiStive bEhaviouR (CARESSER), a novel framework that actively learns robotic assistive behaviour by leveraging the therapist’s expertise (knowledge-driven approach) and their demonstrations (data-driven approach). By exploiting that hybrid approach, the presented method enables in situ fast learning, in a fully autonomous fashion, of personalised patient-specific policies. With the purpose of evaluating our framework, we conducted two user studies in a daily care centre in which older adults affected by mild dementia and mild cognitive impairment (N = 22) were requested to solve cognitive exercises with the support of a therapist and later on of a robot endowed with CARESSER. Results showed that: (i) the robot managed to keep the patients’ performance stable during the sessions even more so than the therapist; (ii) the assistance offered by the robot during the sessions eventually matched the therapist’s preferences. We conclude that CARESSER, with its stakeholder-centric design, can pave the way to new AI approaches that learn by leveraging human–human interactions along with human expertise, which has the benefits of speeding up the learning process, eliminating the need for the design of complex reward functions, and finally avoiding undesired states.Peer ReviewedPostprint (published version

Enhanced Living Environments

This open access book was prepared as a Final Publication of the COST Action IC1303 “Algorithms, Architectures and Platforms for Enhanced Living Environments (AAPELE)”. The concept of Enhanced Living Environments (ELE) refers to the area of Ambient Assisted Living (AAL) that is more related with Information and Communication Technologies (ICT). Effective ELE solutions require appropriate ICT algorithms, architectures, platforms, and systems, having in view the advance of science and technology in this area and the development of new and innovative solutions that can provide improvements in the quality of life for people in their homes and can reduce the financial burden on the budgets of the healthcare providers. The aim of this book is to become a state-of-the-art reference, discussing progress made, as well as prompting future directions on theories, practices, standards, and strategies related to the ELE area. The book contains 12 chapters and can serve as a valuable reference for undergraduate students, post-graduate students, educators, faculty members, researchers, engineers, medical doctors, healthcare organizations, insurance companies, and research strategists working in this area

Human-machine communication for educational systems design

This book contains the papers presented at the NATO Advanced Study Institute (ASI) on the Basics of man-machine communication for the design of educational systems, held August 16-26, 1993, in Eindhoven, The Netherland

A study of novice programmer performance and programming pedagogy.

Identifying and mitigating the difficulties experienced by novice programmers is an active area of research that has embraced a number of research areas. The aim of this research was to perform a holistic study into the causes of poor performance in novice programmers and to develop teaching approaches to mitigate them. A grounded action methodology was adopted to enable the primary concepts of programming cognitive psychology and their relationships to be established, in a systematic and formal manner. To further investigate novice programmer behaviour, two sub-studies were conducted into programming performance and ability. The first sub-study was a novel application of the FP-Tree algorithm to determine if novice programmers demonstrated predictable patterns of behaviour. This was the first study to data mine programming behavioural characteristics rather than the learner’s background information such as age and gender. Using the algorithm, patterns of behaviour were generated and associated with the students’ ability. No patterns of behaviour were identified and it was not possible to predict student results using this method. This suggests that novice programmers demonstrate no set patterns of programming behaviour that can be used determine their ability, although problem solving was found to be an important characteristic. Therefore, there was no evidence that performance could be improved by adopting pedagogies to promote simple changes in programming behaviour beyond the provision of specific problem solving instruction. A second sub-study was conducted using Raven’s Matrices which determined that cognitive psychology, specifically working memory, played an important role in novice programmer ability. The implication was that programming pedagogies must take into consideration the cognitive psychology of programming and the cognitive load imposed on learners. Abstracted Construct Instruction was developed based on these findings and forms a new pedagogy for teaching programming that promotes the recall of abstract patterns while reducing the cognitive demands associated with developing code. Cognitive load is determined by the student’s ability to ignore irrelevant surface features of the written problem and to cross-reference between the problem domain and their mental program model. The former is dealt with by producing tersely written exercises to eliminate distractors, while for the latter the teaching of problem solving should be delayed until the student’s program model is formed. While this does delay the development of problem solving skills, the problem solving abilities of students taught using this pedagogy were found to be comparable with students taught using a more traditional approach. Furthermore, monitoring students’ understanding of these patterns enabled micromanagement of the learning process, and hence explanations were provided for novice behaviour such as difficulties using arrays, inert knowledge and “code thrashing”. For teaching more complex problem solving, scaffolding of practice was investigated through a program framework that could be developed in stages by the students. However, personalising the level of scaffolding required was complicated and found to be difficult to achieve in practice. In both cases, these new teaching approaches evolved as part of a grounded theory study and a clear progression of teaching practice was demonstrated with appropriate evaluation at each stage in accordance with action researc

Cognition, computers and creative writing

This thesis describes a teaching scheme for creative writing that takes account of a child's developing cognitive abilities. It first provides the means for a child to explore language and gain sufficient understanding of linguistic concepts and processes to be able to control the acquisition of new writing skills. This is a preparation for the second part of the scheme, in which a child applies this understanding to her own creative writing. The child is given practice in generating and transforming text at different structural levels, and in selecting text forms that are appropriate to the audience and function of the writing. Computer programs form an integral part of the scheme. They provide representations of two abstract systems - a generative grammar and an associative network - which the child manipulates to investigate language structure and plans. The programs also offer a dynamic medium for text creation and revision. The teaching scheme was tested with six eleven year old children who visited the University for 29 sessions of 60-70 minutes duration, over three school terms. It was presented to the children through written worksheets, containing language exercises, writing activities, and instructions for the use of the computer. The children were set pre and post descriptive and narrative essays. The same essays were also set for a control group of children who followed normal classroom teaching for the period of the project. The main method of assessment was a feature analysis of the essays. They were examined for the occurrence of 'mature' and 'immature' linguistic features, at the word/phrase, sentence, and section level of text. The presence of mature features indicates that the writer is able to reflect on the form of language and can create text to a well-structured and coherent plan. During the first part of the scheme, the children formed into two distinct groups. Two children performed poorly in all activities that involved the understanding and manipulation of language. The four remaining children performed well in these activities and enjoyed the experience of exploring language with the aid of a computer. These four children w^ere taken on to the second stage of the scheme. An analysis of the pre and post essays showed developments in the writing ability of all four children. Each child, however, gained a different set of skills, and appeared to be using the writing projects to explore a particular aspect of style. We suggest that such explorations are valuable, enabling a child to discover the constraints and possibilities of creative writing