A Virtual Conversational Agent for Teens with Autism: Experimental Results and Design Lessons

We present the design of an online social skills development interface for teenagers with autism spectrum disorder (ASD). The interface is intended to enable private conversation practice anywhere, anytime using a web-browser. Users converse informally with a virtual agent, receiving feedback on nonverbal cues in real-time, and summary feedback. The prototype was developed in consultation with an expert UX designer, two psychologists, and a pediatrician. Using the data from 47 individuals, feedback and dialogue generation were automated using a hidden Markov model and a schema-driven dialogue manager capable of handling multi-topic conversations. We conducted a study with nine high-functioning ASD teenagers. Through a thematic analysis of post-experiment interviews, identified several key design considerations, notably: 1) Users should be fully briefed at the outset about the purpose and limitations of the system, to avoid unrealistic expectations. 2) An interface should incorporate positive acknowledgment of behavior change. 3) Realistic appearance of a virtual agent and responsiveness are important in engaging users. 4) Conversation personalization, for instance in prompting laconic users for more input and reciprocal questions, would help the teenagers engage for longer terms and increase the system's utility

Virtual Reality as a Support Tool for the Treatment of People with Intellectual and Multiple Disabilities: A Systematic Literature Review

Since the emergence of virtual reality (VR) technologies, many researchers have argued on the benefits of their use for people with intellectual and multiple disabilities. However, up to this date there is not a single study that presents a detailed overview of the state of the art in virtual reality as a support tool for the treatment of people with intellectual and multiple disabilities, as well as Autism and Down Syndrome. The aim of this study is to provide a detailed overview of the state of the art in the virtual reality area focusing on people with multiple disabilities, that encompasses intellectual and physical disabilities. There is still no consensus on the effectiveness of VR-based treatments. Virtual reality can offer rich environment and features, but most of the researches focuses only in the experience to be inside a virtual place without taking advantage of what benefits VR provide us. Furthermore, most of our selected studies used non-immersive VR and AR. Thus, immersive VR is an open field with many opportunities to be explored. We believe VR has great potential to be effective in the treatment of people with intellectual and multiple disabilities

Cognitive State Measurement from Eye Gaze Analysis in an Intelligent Virtual Reality Driving System for Autism Intervention

Abstract-Autism Spectrum Disorder (ASD) is a group of neurodevelopmental disabilities with a high prevalence rate. While much research has focused on improving social communication deficits in ASD populations, less emphasis has been devoted to improving skills relevant for adult independent living, such as driving. In this paper, a novel virtual reality (VR)-based driving system with different difficulty levels of tasks is presented to train and improve driving skills of teenagers with ASD. The goal of this paper is to measure the cognitive load experienced by an individual with ASD while he is driving in the VR-based driving system. Several eye gaze features are identified that varied with cognitive load in an experiment participated by 12 teenagers with ASD. Several machine learning methods were compared and the ability of these methods to accurately measure cognitive load was validated with respect to the subjective rating of a therapist. Results will be used to build models in an intelligent VR-based driving system that can sense a participant's real-time cognitive load and offer driving tasks at an appropriate difficulty level in order to maximize the participant's long-term performance

Attention-Based Applications in Extended Reality to Support Autistic Users: A Systematic Review

With the rising prevalence of autism diagnoses, it is essential for research to understand how to leverage technology to support the diverse nature of autistic traits. While traditional interventions focused on technology for medical cure and rehabilitation, recent research aims to understand how technology can accommodate each unique situation in an efficient and engaging way. Extended reality (XR) technology has been shown to be effective in improving attention in autistic users given that it is more engaging and motivating than other traditional mediums. Here, we conducted a systematic review of 59 research articles that explored the role of attention in XR interventions for autistic users. We systematically analyzed demographics, study design and findings, including autism screening and attention measurement methods. Furthermore, given methodological inconsistencies in the literature, we systematically synthesize methods and protocols including screening tools, physiological and behavioral cues of autism and XR tasks. While there is substantial evidence for the effectiveness of using XR in attention-based interventions for autism to support autistic traits, we have identified three principal research gaps that provide promising research directions to examine how autistic populations interact with XR. First, our findings highlight the disproportionate geographic locations of autism studies and underrepresentation of autistic adults, evidence of gender disparity, and presence of individuals diagnosed with co-occurring conditions across studies. Second, many studies used an assortment of standardized and novel tasks and self-report assessments with limited tested reliability. Lastly, the research lacks evidence of performance maintenance and transferability.Comment: [Accepted version] K. Wang, S. J. Julier and Y. Cho, "Attention-Based Applications in Extended Reality to Support Autistic Users: A Systematic Review," in IEEE Access, vol. 10, pp. 15574-15593, 2022, doi: 10.1109/ACCESS.2022.314772

HAZARD PERCEPTION TRAINING FOR ADOLESCENTS WITH AUTISM SPECTRUM DISORDER ON THE INTERACTIVE DRIVING SIMULATOR: USING EYE TRACKING TECHNOLOGY TO DETERMINE EFFECTIVENESS

Rationale: Driving is an important developmental milestone for all adolescents as it increases their independence and ability to participate in vehicle-dependent activities. However, adolescents with high functioning autism spectrum disorder (HFASD) are less likely to obtain licenses and drive independently due to characteristics related to their diagnosis. Although current research exists exploring the efficacy of driving simulator training for adolescent drivers with HFASD and eye tracking, there is a gap in the literature related to training on the simulator and its effects on overall driving performance and hazard perception and response in this population. Purpose: This pilot study utilized a training protocol on the simulator that included hazard perception to determine its effect on overall driving performance. Eye tracking technology was used to determine if there was a change in hazard perception and response to non-social and social hazards after training. Design: This study was a one group, pretest-posttest intervention design. Methods: There were 17 participants between the ages of 15 and 22 with a self-reported diagnosis of ASD and a desire to learn to drive independently. Each participant completed a pre-test and post-test on the driving simulator while wearing eye tracking technology. Each participant completed a protocol of 30 learning modules with scenarios related to driving skills and hazard detection and response in one-to-one training. Analysis: Driving performance was measured by a quantitative score from a standardized observational tool for driving. Eye tracking measures including fixation duration, fixation count, and time to first fixation were analyzed using a Wilcoxon Signed Rank Test. Results: Participants significantly increased their overall driving performance scores pre-test to post-test. Results of hazard perception using eye tracking technology tended towards improvement overall, but specific hazard results were inconsistent and varied for both non-social and social hazards in terms of fixation duration, fixation count, and time to first fixation. Discussion: Findings from this study indicate driving simulator training related to hazard perception was effective in improving overall driving simulator performance in adolescents with HFASD. Additionally, findings indicate hazard perception and response differs for this population after hazard perception training, but specific eye tracking measures may increase or decrease, and results may not be specific to non-social or social hazards

Assessment of Sensory Abnormalities in Children with ASD Using Virtual Reality

Sensory abnormality is one of the important characteristics of Autism Spectrum Disorder(ASD). A large number of studies suggest a strong connection between sensory abnormalities and ASD. Therefore, it is important to assess sensory abnormalities for diagnosis and intervention of ASD. In this thesis study, we aim to develop a virtual reality system that can measure sensory abnormalities in adolescents with ASD. Sensory abnormalities can affect all five senses: touch, vision, smell, taste and auditory. However, in this study, we focus on assessing sensory abnormality in vision and touch sensory processing. Twelve adolescents with ASD and 12 typically developing (TD) adolescents aged 11-17 years participated in the study. Participants were assigned a task in which they interacted with the virtual reality system. The system recorded participants’ behavior and their response to the virtual environment in the real time while they completed the task. We defined four measurements to analyze the behavior of the participants. With the help of the defined measurements, we found some significant differences in the way participants with ASD and TD participants interacted with the virtual environment. Participants also filled a commonly used standard psychological assessment questionnaire called Adult/Adolescent Sensory Profile (AASP). Strong correlations between some of the scores in the questionnaire and their response to the virtual environment were also observed. Therefore, this pilot study supports the use of technology to assess sensory abnormalities and shows that further research into the development of such technology is essential

Children on the Autism Spectrum and the Use of Virtual Reality for Supporting Social Skills

Background: Autism spectrum disorders (ASDs) are characterized by differences in socio-pragmatic communication. These conditions are allocated within a “spectrum” of phenotypic variability. Virtual reality (VR) is a useful tool for healthcare intervention and particularly safely advancing social abilities in children with ASD. Methods: In our study two types of intervention for improving social skills were compared: (i) emotional training obtained by the use of virtual reality (Gr1), (ii) traditional emotional training performed individually with a therapist (Gr2). We aimed to identify the intervention with the shortest acquisition time for the proposed social tasks. Results: Our findings show that both types of intervention had the same acquisition time for the recognition of primary emotions. However, for the use of primary and secondary emotions, the group using VR showed shorter acquisition times. Conclusions: These findings together with previous preliminary datasuggest that VR can be a promising, dynamic and effective practice for the support of basic and complex social skills of these individuals

Assessing the Impact of Multi-variate Steering-rate Vehicle Control on Driver Performance in a Simulation Framework

When a driver turns a steering-wheel, he or she normally expects the vehicle\u27s steering system to communicate an equivalent amount of signal to the road-wheels. This relationship is linear and occurs regardless of the steering-wheel\u27s position within its rotational travel. The linear steering paradigm in passenger vehicles has gone largely unchanged since mass production of passenger vehicles began in 1901. However, as more electronically-controlled steering systems appear in conjunction with development of autonomous steering functions in vehicles, an opportunity to advance the existing steering paradigms arises. The following framework takes a human-factors approach toward examining and evaluating alternative steering systems by using Modeling and Simulation methods to track and score human performance. Present conventional steering systems apply a linear relationship between the steering-wheel and the road wheels of a vehicle. The rotational travel of the steering-wheel is 900° and requires two-and-a-half revolutions to travel from end-stop to opposite end-stop. The experimental steering system modeled and employed in this study applies a dynamic curve response to the steering input within a shorter, 225° rotational travel. Accommodation variances, based on vehicle speed and steering-wheel rotational position and acceleration, moderate the apparent steering input to augment a more-practical, effective steering rate. This novel model follows a paradigm supporting the full range of steering-wheel actuation without necessitating hand repositioning or the removal of the driver\u27s hands from the steering-wheel during steering maneuvers. In order to study human performance disparities between novel and conventional steering models, a custom simulator was constructed and programmed to render representative models in a test scenario. Twenty-seven males and twenty-seven females, ranging from the ages of eighteen to sixty-five were tested and scored using the driving simulator that presented two successive driving test vignettes: One vignette using conventional 900° steering with linear response and the other employing the augmented 225° multivariate, non-linear steering. The results from simulator testing suggest that both males and females perform better with the novel system, supporting the hypothesis that drivers of either gender perform better with a system augmented with 225° multivariate, non-linear steering than with a conventional steering system. Further analysis of the simulated-driving scores indicates performance parity between male and female participants, supporting the hypothesis positing no significant difference in driver performance between male and female drivers using the augmented steering system. Finally, composite data from written questionnaires support the hypothesis that drivers will prefer driving the augmented system over conventional steering. These collective findings support justification for testing and refining novel steering systems using Modeling and Simulation methods. As a product of this particular study, a tested and open-sourced simulation framework now exists such that researchers and automotive designers can develop, as well as evaluate their own steering-oriented products within a valid human-factors construct. The open-source nature of this framework implies a commonality by which otherwisedisparate research and development work can be associated. Extending this framework beyond basic investigation to reach applications requiring morespecialized parameters may even impact drivers having special needs. For example, steeringsystem functional characteristics could be comparatively optimized to accommodate individuals afflicted with upper-body deficits or limited use of either or both arms. Moreover, the combined human-factors and open-source approaches distinguish the products of this research as a common and extensible platform by which purposeful automotive-industry improvements can be realized—contrasted with arbitrary improvements that might be brought about predominantly to showcase technological advancements