Human-centred design methods : developing scenarios for robot assisted play informed by user panels and field trials

Original article can be found at: http://www.sciencedirect.com/ Copyright ElsevierThis article describes the user-centred development of play scenarios for robot assisted play, as part of the multidisciplinary IROMEC1 project that develops a novel robotic toy for children with special needs. The project investigates how robotic toys can become social mediators, encouraging children with special needs to discover a range of play styles, from solitary to collaborative play (with peers, carers/teachers, parents, etc.). This article explains the developmental process of constructing relevant play scenarios for children with different special needs. Results are presented from consultation with panel of experts (therapists, teachers, parents) who advised on the play needs for the various target user groups and who helped investigate how robotic toys could be used as a play tool to assist in the children’s development. Examples from experimental investigations are provided which have informed the development of scenarios throughout the design process. We conclude by pointing out the potential benefit of this work to a variety of research projects and applications involving human–robot interactions.Peer reviewe

Psychophysiological analysis of a pedagogical agent and robotic peer for individuals with autism spectrum disorders.

Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by ongoing problems in social interaction and communication, and engagement in repetitive behaviors. According to Centers for Disease Control and Prevention, an estimated 1 in 68 children in the United States has ASD. Mounting evidence shows that many of these individuals display an interest in social interaction with computers and robots and, in general, feel comfortable spending time in such environments. It is known that the subtlety and unpredictability of people’s social behavior are intimidating and confusing for many individuals with ASD. Computerized learning environments and robots, however, prepare a predictable, dependable, and less complicated environment, where the interaction complexity can be adjusted so as to account for these individuals’ needs. The first phase of this dissertation presents an artificial-intelligence-based tutoring system which uses an interactive computer character as a pedagogical agent (PA) that simulates a human tutor teaching sight word reading to individuals with ASD. This phase examines the efficacy of an instructional package comprised of an autonomous pedagogical agent, automatic speech recognition, and an evidence-based instructional procedure referred to as constant time delay (CTD). A concurrent multiple-baseline across-participants design is used to evaluate the efficacy of intervention. Additionally, post-treatment probes are conducted to assess maintenance and generalization. The results suggest that all three participants acquired and maintained new sight words and demonstrated generalized responding. The second phase of this dissertation describes the augmentation of the tutoring system developed in the first phase with an autonomous humanoid robot which serves the instructional role of a peer for the student. In this tutoring paradigm, the robot adopts a peer metaphor, where its function is to act as a peer. With the introduction of the robotic peer (RP), the traditional dyadic interaction in tutoring systems is augmented to a novel triadic interaction in order to enhance the social richness of the tutoring system, and to facilitate learning through peer observation. This phase evaluates the feasibility and effects of using PA-delivered sight word instruction, based on a CTD procedure, within a small-group arrangement including a student with ASD and the robotic peer. A multiple-probe design across word sets, replicated across three participants, is used to evaluate the efficacy of intervention. The findings illustrate that all three participants acquired, maintained, and generalized all the words targeted for instruction. Furthermore, they learned a high percentage (94.44% on average) of the non-target words exclusively instructed to the RP. The data show that not only did the participants learn nontargeted words by observing the instruction to the RP but they also acquired their target words more efficiently and with less errors by the addition of an observational component to the direct instruction. The third and fourth phases of this dissertation focus on physiology-based modeling of the participants’ affective experiences during naturalistic interaction with the developed tutoring system. While computers and robots have begun to co-exist with humans and cooperatively share various tasks; they are still deficient in interpreting and responding to humans as emotional beings. Wearable biosensors that can be used for computerized emotion recognition offer great potential for addressing this issue. The third phase presents a Bluetooth-enabled eyewear – EmotiGO – for unobtrusive acquisition of a set of physiological signals, i.e., skin conductivity, photoplethysmography, and skin temperature, which can be used as autonomic readouts of emotions. EmotiGO is unobtrusive and sufficiently lightweight to be worn comfortably without interfering with the users’ usual activities. This phase presents the architecture of the device and results from testing that verify its effectiveness against an FDA-approved system for physiological measurement. The fourth and final phase attempts to model the students’ engagement levels using their physiological signals collected with EmotiGO during naturalistic interaction with the tutoring system developed in the second phase. Several physiological indices are extracted from each of the signals. The students’ engagement levels during the interaction with the tutoring system are rated by two trained coders using the video recordings of the instructional sessions. Supervised pattern recognition algorithms are subsequently used to map the physiological indices to the engagement scores. The results indicate that the trained models are successful at classifying participants’ engagement levels with the mean classification accuracy of 86.50%. These models are an important step toward an intelligent tutoring system that can dynamically adapt its pedagogical strategies to the affective needs of learners with ASD

Combining PEAK (Promoting the Emergence of Advanced Knowledge) with Other Teaching Methodologies with Children with and without Learning Delays

Promoting the Emergence of Advanced Knowledge (PEAK) combines relational learning and applied behaviour analysis in a teaching programme for children with diagnosed autism (Dixon, 2014). Study 1a aimed to expand on this research by examining PEAK Sequencing; if teaching learning targets taken from the PEAK Generalisation Module will negate having to teach learning targets in the more basic PEAK Direct Module (emergent learning) with preschool children (n=8) both typically developing (4) and diagnosed with autism (4). Pre and post-training measures of standardised ability; Peabody Picture Vocabulary Test (PPVT-IV; Dunn & Dunn, 2007) and Bracken School Readiness Assessment (BSRA; Bracken, 2007) were compared to determine any positive impact on participant scores, and data were analysed using single-subject design and within- participant data. Study 1a taught 20 PEAK targets using the PEAK Generalisation Module and when tested, 34 emergent skills, previously scored as not in the child’s repertoire when assessed on the PEAK Direct module were mastered. The second part, Study 1b, further aimed to combine PEAK with an interactive computerised teaching programme (T-IRAP) to facilitate 'fluent' (rapid and accurate) participant responding with previously taught (study 1a) PEAK targets such as ‘matching’ and ‘exclusion’ were taught using the T-IRAP using ‘same/different’ relations. The study successfully combined PEAK-ABA with the interactive computerised teaching program, TIRAP. Study 2 successfully taught 5 PEAK targets to preschool children (n=3) with speech and language delay and combined PEAK targets with the Rapid Automatic Naming (RAN; Denckla & Rudel, 1976b) for fluency training. Pre and post assessments, BSRA-3 and PPVTIV were carried out for participants. Results show participants mastered PEAK targets

The Effect of Music Therapy and Peer-Mediated Interventions on Social-Communicative Responses of Children with Autism Spectrum Disorders

The purpose of this study was to examine the effect of music therapy on social-communicative behaviors (eye contact, vocalization/verbalization, and gesture imitation) of children with autism spectrum disorders. The target participants were two children with a tentative diagnosis or a diagnosis of autism spectrum disorders along with three typically developing children. ABAB reversal design was used to investigate the difference in the social-communicative behaviors between baseline (A) and music therapy intervention (B). The frequency of social-communicative behaviors of eye contact, vocalization/verbalization, and gestural imitation were reported through graphic analysis. In each condition, one or two typically developing children modeled the social-communicative behaviors for their peers with autism spectrum disorders. Results from the two participants indicated that the use of either intervention (non music or music) could increase one of the social-communicative behaviors in children with autism spectrum disorders, gestural imitation. As for vocalization/verbalization, the two target participants demonstrated this behavior more during the non-music condition than during the music condition. Eye contact results indicated that there was no different between the non-music and the music conditions

Enhancing the Application and Evaluation of a Discrete Trial Intervention Package for Eliciting First Words in Preverbal Preschoolers with ASD

This study evaluates the effectiveness of an intervention package including a discrete trial program (Rapid Motor Imitation Antecedent Training (Tsiouri and Greer, J Behav Educat 12:185–206, 2003) combined with parent education for eliciting first words in children with ASD who had little or no spoken language. Evaluation of the approach includes specific intervention targets and functional spoken language outcomes (Tager-Flusberg et al., J Speech Lang Hear Res 52:643–652, 2009). Results suggest that RMIA, with parent training, catalyzes development of verbal imitation and production for some children. Three of five participants acquired word production within the DTT framework and achieved milestones of early functional spoken language use (Tager-Flusberg et al., J Speech Lang Hear Res 52:643–652, 2009). The implications of these findings for understanding the role of discrete trial approaches to language intervention are discussed

A Review of “Music and Movement” Therapies for Children with Autism: Embodied Interventions for Multisystem Development

The rising incidence of Autism Spectrum Disorders (ASDs) has led to a surge in the number of children needing autism interventions. This paper is a call to clinicians to diversify autism interventions and to promote the use of embodied music-based approaches to facilitate multisystem development. Approximately 12% of all autism interventions and 45% of all alternative treatment strategies in schools involve music-based activities. Musical training impacts various forms of development including communication, social-emotional, and motor development in children with ASDs and other developmental disorders as well as typically developing children. In this review, we will highlight the multisystem impairments of ASDs, explain why music and movement therapies are a powerful clinical tool, as well as describe mechanisms and offer evidence in support of music therapies for children with ASDs. We will support our claims by reviewing results from brain imaging studies reporting on music therapy effects in children with autism. We will also discuss the critical elements and the different types of music therapy approaches commonly used in pediatric neurological populations including autism. We provide strong arguments for the use of music and movement interventions as a multisystem treatment tool for children with ASDs. Finally, we also make recommendations for assessment and treatment of children with ASDs, and provide directions for future research

Evaluation of a Computer-Based Observer-Effect Training on Mothers\u27 Vocal Imitation of Their Infant

Infants begin to learn important skills, such as contingency learning, social referencing, and joint attention through everyday interactions with their environment. When infants learn that their behavior produces a change in the environment (e.g., attention from others), infants engage in behavior that produces that effect (e.g., increases in smiling sustained engagement. When mothers and other caregivers respond immediately to infant behavior, they help their infant learn that the infant’s own behavior is effective, producing a change in the environment. The current investigation evaluated the effect of a computer-based training that aimed at teaching mothers to play a vocal-imitation contingency-learning game. The training included observer-effect methodology, meaning the mothers engaged in observation and evaluation of other mothers engaging in vocal imitation but did not themselves receive any direct coaching or feedback. All mothers completed the training during one session and in less than 45 min. Results indicate that all mothers increased their use of vocal imitation post training and maintained their performance at a two-week follow-up. Results are discussed in terms of how computer training may facilitate dissemination of responsive caregiver training

Charlie: A New Robot Prototype for Improving Communication and social Skills in Children with Autism and a New Single-point Infrared Sensor Technique for Detecting bBeathing and Heart Rate Remotely

This research delivers a new, interactive game-playing robot named CHARLIE and a novel technique for remotely detecting breathing and heart rate using a single-point, thermal infrared sensor (IR). The robot is equipped with a head and two arms, each with two degrees of freedom, and a camera. We trained a human hands classifier and used this classifier along with a standard face classifier to create two autonomous interactive games: single-player ( Imitate Me, Imitate You ) and two-player ( Pass the Pose ). Further, we developed and implemented a suite of new interactive games in which the robot is teleoperated by remote control. Each of these features has been tested and validated through a field study including eight children diagnosed with autism and speech delays. Results from that study show that significant improvements in speech and social skills can be obtained when using CHARLIE with the methodology described herein. Moreover, gains in communication and social interaction are observed to generalize from child-to-robot to co-present others through the scaffolding of communication skills with the systematic approach developed for the study. Additionally, we present a new IR system that continuously targets the sub-nasal region of the face and measures subtle temperature changes corresponding to breathing and cardiac pulse. This research makes four novel contributions: (1) A low-cost, field-tested robot for use in autism therapy, (2) a suite of interactive robot games, (3) a hand classifier created for performing hand detection during the interactive games, and (4) an IR sensor system which remotely collects temperatures and computes breathing and heart rate. Interactive robot CHARLIE is physically designed to be aesthetically appealing to young children between three and six years of age. The hard, wood and metal robot body is covered with a bright green, fuzzy material and additional padding so that it appears toylike and soft. Additionally, several structural features were included to ensure safety during interactive play and to enhance the robustness of the robot. Because children with autism spectrum disorder (ASD) often enjoy exploring new or interesting objects with their hands, the robot must be able to withstand a moderate amount of physical manipulation without causing injury to the child or damaging the robot or its components. CHARLIE plays five distinct interactive games that are designed to be entertaining to young children, appeal to children of varying developmental ability and promote increased speech and social skill through imitation and turn-taking. Remote breathing and heart rate detection Stress is a compounding factor in autism therapy which can inhibit progress toward specific therapeutic goals. The ability to non-invasively detect physical indicators of increasing stress, especially when they can be correlated to specific activities and measured in terms of length and frequency, can relay important metrics about the antecedents that cause stress for a particular child and can be used to help automate the evaluation of a child\u27s progress between sessions. Further, collecting and measuring critical physiological indicators such as breathing and heart rate can enable robots to adjust their behavior based on the perceived emotional, psychological or physical state of their user. The utility and acceptance of robots can be further increased when they are able to learn typical physiological patterns and use these patterns as a baseline for identifying anomalies or possible warning signs of various problems in their human users. We present a new technique for remotely collecting and analyzing breathing and heart rates in real time using an autonomous, low cost infrared (IR) sensor system. This is accomplished by continuously targeting a high precision IR sensor, tracking changes in the sub-nasal skin surface temperature and employing a sinusoidal curve-fitting function, Fast Fourier Transform (FFT), and Discrete Wavelet Transform (DWT) to extract the breathing and heart rate from recorded temperatures