Psychophysiological analysis of a pedagogical agent and robotic peer for individuals with autism spectrum disorders.

Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by ongoing problems in social interaction and communication, and engagement in repetitive behaviors. According to Centers for Disease Control and Prevention, an estimated 1 in 68 children in the United States has ASD. Mounting evidence shows that many of these individuals display an interest in social interaction with computers and robots and, in general, feel comfortable spending time in such environments. It is known that the subtlety and unpredictability of people’s social behavior are intimidating and confusing for many individuals with ASD. Computerized learning environments and robots, however, prepare a predictable, dependable, and less complicated environment, where the interaction complexity can be adjusted so as to account for these individuals’ needs. The first phase of this dissertation presents an artificial-intelligence-based tutoring system which uses an interactive computer character as a pedagogical agent (PA) that simulates a human tutor teaching sight word reading to individuals with ASD. This phase examines the efficacy of an instructional package comprised of an autonomous pedagogical agent, automatic speech recognition, and an evidence-based instructional procedure referred to as constant time delay (CTD). A concurrent multiple-baseline across-participants design is used to evaluate the efficacy of intervention. Additionally, post-treatment probes are conducted to assess maintenance and generalization. The results suggest that all three participants acquired and maintained new sight words and demonstrated generalized responding. The second phase of this dissertation describes the augmentation of the tutoring system developed in the first phase with an autonomous humanoid robot which serves the instructional role of a peer for the student. In this tutoring paradigm, the robot adopts a peer metaphor, where its function is to act as a peer. With the introduction of the robotic peer (RP), the traditional dyadic interaction in tutoring systems is augmented to a novel triadic interaction in order to enhance the social richness of the tutoring system, and to facilitate learning through peer observation. This phase evaluates the feasibility and effects of using PA-delivered sight word instruction, based on a CTD procedure, within a small-group arrangement including a student with ASD and the robotic peer. A multiple-probe design across word sets, replicated across three participants, is used to evaluate the efficacy of intervention. The findings illustrate that all three participants acquired, maintained, and generalized all the words targeted for instruction. Furthermore, they learned a high percentage (94.44% on average) of the non-target words exclusively instructed to the RP. The data show that not only did the participants learn nontargeted words by observing the instruction to the RP but they also acquired their target words more efficiently and with less errors by the addition of an observational component to the direct instruction. The third and fourth phases of this dissertation focus on physiology-based modeling of the participants’ affective experiences during naturalistic interaction with the developed tutoring system. While computers and robots have begun to co-exist with humans and cooperatively share various tasks; they are still deficient in interpreting and responding to humans as emotional beings. Wearable biosensors that can be used for computerized emotion recognition offer great potential for addressing this issue. The third phase presents a Bluetooth-enabled eyewear – EmotiGO – for unobtrusive acquisition of a set of physiological signals, i.e., skin conductivity, photoplethysmography, and skin temperature, which can be used as autonomic readouts of emotions. EmotiGO is unobtrusive and sufficiently lightweight to be worn comfortably without interfering with the users’ usual activities. This phase presents the architecture of the device and results from testing that verify its effectiveness against an FDA-approved system for physiological measurement. The fourth and final phase attempts to model the students’ engagement levels using their physiological signals collected with EmotiGO during naturalistic interaction with the tutoring system developed in the second phase. Several physiological indices are extracted from each of the signals. The students’ engagement levels during the interaction with the tutoring system are rated by two trained coders using the video recordings of the instructional sessions. Supervised pattern recognition algorithms are subsequently used to map the physiological indices to the engagement scores. The results indicate that the trained models are successful at classifying participants’ engagement levels with the mean classification accuracy of 86.50%. These models are an important step toward an intelligent tutoring system that can dynamically adapt its pedagogical strategies to the affective needs of learners with ASD

Implementing Gameplay Skills to Increase Eye Contact and Communication for Students with Emotional Behavioral Disorder and Comorbid Disabilities

This study was conducted to examine the effectiveness of gameplay activities using a structured social skills program to increase both eye contact responses and the number of verbal responses during peer relationships for students with comorbid disabilities in a clinical setting. This SEL intervention was modified for children with comorbid autism spectrum disorder (ASD) and emotional and behavioral disorder (EBD). A single-subject, multiple-baseline, across-participants design was used. Participants included adolescents (n = 6) with comorbid disabilities, predominantly ASD and EBD, their board-certified behavior analysts (BCBA), and six behavior technicians. The researcher established and maintained face-to-face reciprocal peer social engagements (communication) and direct eye gaze (contact) with participants in dyad groups. Each session included a 60-minute video of peer social interactions. Direct observations, differential reinforcement of other behaviors (DRO), and pre- and post-Social Skills Improvement System-Rating Scales (SSIS-RS) were examined. Evidence of SEL intervention effectiveness was measured by percentage of nonoverlapping data points (PND). Social validity was measured using the multiple-rater SSIS-RS and intervention fidelity checklists evaluating the Sanford Harmony intervention. Results from data and visual analysis revealed all participants significantly increased their direct eye contact, verbal reciprocity, and social engagements after implementing the Harmony program. In addition, a PND value of 100% was calculated for each dependent variable indicating the Harmony program was a highly effective intervention increasing eye contact, verbal reciprocity, and social engagements for students with EBD and comorbid disabilities

A proposal to act on Theory of Mind by applying robotics and virtual worlds with children with ASD

The article proposes an intervention framed under a single-subject research design where robotics and a 3D virtual environment are used jointly to improve the development of Theory of Mind in children with ASD. The project aims at verifying if the use of a humanoid robot, with high interactive abilities and responses, along with a virtual robot in a social virtual world can enable an improved comprehension of emotions and perspective taking. Specifically the planned activities are designed to gradually support the subjects with ASD in interactional settings in order to make them acquire the needed self-confidence to finally interact with a classmate in the virtual environment

Applications of Robotics for Autism Spectrum Disorder: a Scoping Review

Robotic therapies are receiving growing interest in the autism field, especially for the improvement of social skills of children, enhancing traditional human interventions. In this work, we conduct a scoping review of the literature in robotics for autism, providing the largest review on this field from the last five years. Our work underlines the need to better characterize participants and to increase the sample size. It is also important to develop homogeneous training protocols to analyse and compare the results. Nevertheless, 7 out of the 10 Randomized control trials reported a significant impact of robotic therapy. Overall, robot autonomy, adaptability and personalization as well as more standardized outcome measures were pointed as the most critical issues to address in future research

Design of a robotic toy and user interfaces for autism spectrum disorder risk assessment

Autism Spectrum Disorder (ASD) is an umbrella term for a spectrum of complex developmental disorders resulting in deficits in social communication and repetitive and stereotyped behaviors. According to research conducted in 2014, one in every 68 children in the United States is diagnosed with ASD. Despite this observation, there is no national screening system in Turkey, and screenings are not conducted systematically. Research in the area revealed that individuals with ASD are more interested in interact with technology (e.g. computers, iPad, robots, etc.) than human beings. This thesis includes research into how to design and use technology to create suitable products for deficits of ASD. With all of the concern over the high prevalence ratios of ASD, this thesis presents the methodology and design of a risk assessment device, which aims to capture the interest of children with ASD aged 3-4, and direct children who score low on the tests towards a diagnosis. In particular, the tests in the device focus on Theory of Mind (ToM) development and designed to detect differences with ToM tests between ASD and Typically Developing (TD) children. In the scope of the thesis, 2D illustrations, interface design, and outer shell design of the device are created in compliance with the research data in the field. Finally, outer shell design is 3D printed, and then surface is sanded and spray-painted

Bridging the Research Gap: Making HRI Useful to Individuals with Autism

While there is a rich history of studies involving robots and individuals with autism spectrum disorders (ASD), few of these studies have made substantial impact in the clinical research community. In this paper we first examine how differences in approach, study design, evaluation, and publication practices have hindered uptake of these research results. Based on ten years of collaboration, we suggest a set of design principles that satisfy the needs (both academic and cultural) of both the robotics and clinical autism research communities. Using these principles, we present a study that demonstrates a quantitatively measured improvement in human-human social interaction for children with ASD, effected by interaction with a robot

A Music-Therapy Robotic Platform for Children with Autism: A Pilot Study

Children with Autism Spectrum Disorder (ASD) experience deficits in verbal and nonverbal communication skills including motor control, turn-taking, and emotion recognition. Innovative technology, such as socially assistive robots, has shown to be a viable method for Autism therapy. This paper presents a novel robot-based music-therapy platform for modeling and improving the social responses and behaviors of children with ASD. Our autonomous social interactive system consists of three modules. Module one provides an autonomous initiative positioning system for the robot, NAO, to properly localize and play the instrument (Xylophone) using the robot’s arms. Module two allows NAO to play customized songs composed by individuals. Module three provides a real-life music therapy experience to the users. We adopted Short-time Fourier Transform and Levenshtein distance to fulfill the design requirements: 1) “music detection” and 2) “smart scoring and feedback”, which allows NAO to understand music and provide additional practice and oral feedback to the users as applicable. We designed and implemented six Human-Robot-Interaction (HRI) sessions including four intervention sessions. Nine children with ASD and seven Typically Developing participated in a total of fifty HRI experimental sessions. Using our platform, we collected and analyzed data on social behavioral changes and emotion recognition using Electrodermal Activity (EDA) signals. The results of our experiments demonstrate most of the participants were able to complete motor control tasks with 70% accuracy. Six out of the nine ASD participants showed stable turn-taking behavior when playing music. The results of automated emotion classification using Support Vector Machines illustrates that emotional arousal in the ASD group can be detected and well recognized via EDA bio-signals. In summary, the results of our data analyses, including emotion classification using EDA signals, indicate that the proposed robot-music based therapy platform is an attractive and promising assistive tool to facilitate the improvement of fine motor control and turn-taking skills in children with ASD

Effect of sensory-based technologies on atypical sensory responses of children with Autism Spectrum Disorder: A systematic review

© 2021 ACM, Inc. This is the accepted manuscript version of an article which has been published in final form at https://doi.org/10.1145/3485768.3485782.Atypical sensory responses are one of the most common issues observed in Autism Spectrum Disorder (ASD), affecting the development of a child's capability for social interaction, independent living and learning. In the past two decades, there has been a growing number of studies of technology-based interventions for atypical sensory responses of individuals with ASD. However, their effects and limitations have not been fully examined. This systematic review investigates the effects of sensory-based technologies (SBTs) on atypical sensory responses of children with ASD. Publications that report on the use of a SBT as an intervention tool were retrieved from four academic databases: “PubMed”, “IEEE Xplore”, “ACM Digital Library” and “Web of Science”. The search finally yielded 18 articles. The results indicated an emerging trend of studies investigating the effects of SBTs on atypical sensory responses over the past decade. Challenges and limitations were found in studies, mainly because the literatures adopted different methods and indicators, small sample sizes, and varying experimental designs. Findings were that the use of SBTs could effectively improve auditory and visual recognition, and some other behavioural outcomes such as attention in children with ASD. Future development of SBTs could further integrate more advanced techniques, such as machine learning, in order to widen the scope of SBTs usage to help more ASD children