Virtual reality based behavioural learning for autistic children

Autism is a disorder in the growth and development of a brain or central nervous system that covers a large spectrum of impairment, symptoms and skills. The children who are suffering from autism face difficulties in communicating and adapting well in the community as they have trouble in understanding what others think and feel. Therefore, there is a need to design effective e-learning method to ease the communication process and to deliver required knowledge to autistic children. Past researchers have highlighted that a virtual reality based learning environment, a computer simulated environment, can facilitate the learning process among autistic children. It is also recognized that the virtual agent plays an important role in virtual worlds as it eases the communication process between the virtual environment (VE) and children with autism. This research aimed to design an effective learning environment for autistic children by developing a virtual environment prototype using face-to-face interviews and picture exchange communication system (PECS) methodology for data collection which was analysed using quantitative tests. The findings suggest that the toilet virtual environment topped the list for being the most popular learning environment among autistic children for behavioural training. The designed prototype identifies autistic children’s and their parents’ needs and also addresses limitations in an existing virtual environment

Multimodal agents for cooperative interaction

2020 Fall.Includes bibliographical references.Embodied virtual agents offer the potential to interact with a computer in a more natural manner, similar to how we interact with other people. To reach this potential requires multimodal interaction, including both speech and gesture. This project builds on earlier work at Colorado State University and Brandeis University on just such a multimodal system, referred to as Diana. I designed and developed a new software architecture to directly address some of the difficulties of the earlier system, particularly with regard to asynchronous communication, e.g., interrupting the agent after it has begun to act. Various other enhancements were made to the agent systems, including the model itself, as well as speech recognition, speech synthesis, motor control, and gaze control. Further refactoring and new code were developed to achieve software engineering goals that are not outwardly visible, but no less important: decoupling, testability, improved networking, and independence from a particular agent model. This work, combined with the effort of others in the lab, has produced a "version 2'' Diana system that is well positioned to serve the lab's research needs in the future. In addition, in order to pursue new research opportunities related to developmental and intervention science, a "Faelyn Fox'' agent was developed. This is a different model, with a simplified cognitive architecture, and a system for defining an experimental protocol (for example, a toy-sorting task) based on Unity's visual state machine editor. This version too lays a solid foundation for future research

A closer look at gaze

In this paper, we describe the formatting guidelines for ACM SI

Emoty: an Emotionally Sensitive Conversational Agent for People with Neurodevelopmental Disorders

Our research aims at exploiting the advances in conversational technology to support people with Neurodevelopmental Disorder (NDD). NDD is a group of conditions that are characterized by severe deficits in the cognitive, emotional and motor areas and produce severe impairments in communication and social functioning. This paper presents the design, technology and exploratory evaluation of Emoty, a spoken Conversational Agent (CA) created specifically for individuals with NDD. The goal of Emoty is to help these persons enhancing communication abilities related to emotional recognition and expression, which are fundamental in any form of human relationship. The system exploits emotion detection capabilities based on the semantics of the speech by calling the IBM Watson Tone Analyzer API and from the harmonic features of the audio thanks to an “all-of-us” Deep Learning model. The design and evaluation of Emoty are based on the close collaboration among computer engineers and specialists in NDD (psychologists, neurological doctors, educators)

Teaching children with autism spectrum disorder with restricted interests : a review of evidence for best practice

Inclusive education requires teachers to adapt to children’s learning styles. Children with autism spectrum disorder bring challenges to classroom teach- ing, often exhibiting interests restricted to particular topics. Teachers can be faced with a dilemma either to accommodate these restricted interests (RIs) into teaching or to keep them out of the classroom altogether. In this article, we examined all peer-reviewed studies of teaching children with autism spec- trum disorder with RIs published between 1990 and 2014. We find that posi- tive gains in learning and social skills can be achieved by incorporating children’s RIs into classroom practice: Of 20 published studies that examined 91 children, all reported gains in educational attainment and/or social engagement. Negative consequences were limited to a decrease in task per- formance in one child and a transient increase in perseverative behaviors in two children. The evidence supports the inclusion of RIs into classroom prac- tice. Methods of inclusion of RIs are discussed in light of practical difficulties and ideal outcomes

Effectiveness of Virtual Reality for Children and Adolescents with Autism Spectrum Disorder: An Evidence-Based Systematic Review

[EN] Autism Spectrum Disorder (ASD) is a neurodevelopmental disease that is specially characterized by impairments in social communication and social skills. ASD has a high prevalence in children, affecting 1 in 160 subjects. Virtual reality (VR) has emerged as an effective tool for intervention in the health field. Different recent papers have reviewed the VR-based treatments in ASD, but they have an important limitation because they only use clinical databases and do not include important technical indexes such as the Web of Science index or the Scimago Journal & Country Rank. To our knowledge, this is the first contribution that has carried out an evidence-based systematic review including both clinical and technical databases about the effectiveness of VR-based intervention in ASD. The initial search identified a total of 450 records. After the exclusion of the papers that are not studies, duplicated articles, and the screening of the abstract and full text, 31 articles met the PICO (Population, Intervention, Comparison and Outcomes) criteria and were selected for analysis. The studies examined suggest moderate evidence about the effectiveness of VR-based treatments in ASD. VR can add many advantages to the treatment of ASD symptomatology, but it is necessary to develop consistent validations in future studies to state that VR can effectively complement the traditional treatments.Mesa Gresa, P.; Gil Gómez, H.; Lozano Quilis, JA.; Gil-Gómez, J. (2018). Effectiveness of Virtual Reality for Children and Adolescents with Autism Spectrum Disorder: An Evidence-Based Systematic Review. Sensors. 18(8):1-15. https://doi.org/10.3390/s18082486S115188World Health Organizationhttp://www.who.int/en/news-room/fact-sheets/detail/autism-spectrum-disordersColombi, C., & Ghaziuddin, M. (2017). Neuropsychological Characteristics of Children with Mixed Autism and ADHD. Autism Research and Treatment, 2017, 1-5. doi:10.1155/2017/5781781Merriam-Websterhttps://www.merriam-webster.com/dictionary/virtual%20realityBird, M.-L., Cannell, J., Jovic, E., Rathjen, A., Lane, K., Tyson, A., … Smith, S. (2017). A Randomized Controlled Trial Investigating the Efficacy of Virtual Reality in Inpatient Stroke Rehabilitation. Archives of Physical Medicine and Rehabilitation, 98(10), e27. doi:10.1016/j.apmr.2017.08.084Albiol-Pérez, S., Gil-Gómez, J.-A., Muñoz-Tomás, M.-T., Gil-Gómez, H., Vial-Escolano, R., & Lozano-Quilis, J.-A. (2017). The Effect of Balance Training on Postural Control in Patients with Parkinson’s Disease Using a Virtual Rehabilitation System. Methods of Information in Medicine, 56(02), 138-144. doi:10.3414/me16-02-0004Garcia-Palacios, A., Herrero, R., Vizcaíno, Y., Belmonte, M. A., Castilla, D., Molinari, G., … Botella, C. (2015). Integrating Virtual Reality With Activity Management for the Treatment of Fibromyalgia. The Clinical Journal of Pain, 31(6), 564-572. doi:10.1097/ajp.0000000000000196Bekelis, K., Calnan, D., Simmons, N., MacKenzie, T. A., & Kakoulides, G. (2017). Effect of an Immersive Preoperative Virtual Reality Experience on Patient Reported Outcomes. Annals of Surgery, 265(6), 1068-1073. doi:10.1097/sla.0000000000002094Orlosky, J., Itoh, Y., Ranchet, M., Kiyokawa, K., Morgan, J., & Devos, H. (2017). Emulation of Physician Tasks in Eye-Tracked Virtual Reality for Remote Diagnosis of Neurodegenerative Disease. IEEE Transactions on Visualization and Computer Graphics, 23(4), 1302-1311. doi:10.1109/tvcg.2017.2657018Areces, D., Rodríguez, C., García, T., Cueli, M., & González-Castro, P. (2016). Efficacy of a Continuous Performance Test Based on Virtual Reality in the Diagnosis of ADHD and Its Clinical Presentations. Journal of Attention Disorders, 22(11), 1081-1091. doi:10.1177/1087054716629711Phé, V., Cattarino, S., Parra, J., Bitker, M.-O., Ambrogi, V., Vaessen, C., & Rouprêt, M. (2016). Outcomes of a virtual-reality simulator-training programme on basic surgical skills in robot-assisted laparoscopic surgery. The International Journal of Medical Robotics and Computer Assisted Surgery, 13(2), e1740. doi:10.1002/rcs.1740Pulijala, Y., Ma, M., Pears, M., Peebles, D., & Ayoub, A. (2018). Effectiveness of Immersive Virtual Reality in Surgical Training—A Randomized Control Trial. Journal of Oral and Maxillofacial Surgery, 76(5), 1065-1072. doi:10.1016/j.joms.2017.10.002Jarrold, W., Mundy, P., Gwaltney, M., Bailenson, J., Hatt, N., McIntyre, N., … Swain, L. (2013). Social Attention in a Virtual Public Speaking Task in Higher Functioning Children With Autism. Autism Research, 6(5), 393-410. doi:10.1002/aur.1302Mishkind, M. C., Norr, A. M., Katz, A. C., & Reger, G. M. (2017). Review of Virtual Reality Treatment in Psychiatry: Evidence Versus Current Diffusion and Use. Current Psychiatry Reports, 19(11). doi:10.1007/s11920-017-0836-0Liu, X., Wu, Q., Zhao, W., & Luo, X. (2017). Technology-Facilitated Diagnosis and Treatment of Individuals with Autism Spectrum Disorder: An Engineering Perspective. Applied Sciences, 7(10), 1051. doi:10.3390/app7101051Van Bennekom, M. J., de Koning, P. P., & Denys, D. (2017). Virtual Reality Objectifies the Diagnosis of Psychiatric Disorders: A Literature Review. Frontiers in Psychiatry, 8. doi:10.3389/fpsyt.2017.00163Provoost, S., Lau, H. M., Ruwaard, J., & Riper, H. (2017). Embodied Conversational Agents in Clinical Psychology: A Scoping Review. Journal of Medical Internet Research, 19(5), e151. doi:10.2196/jmir.6553Lau, H. M., Smit, J. H., Fleming, T. M., & Riper, H. (2017). Serious Games for Mental Health: Are They Accessible, Feasible, and Effective? A Systematic Review and Meta-analysis. Frontiers in Psychiatry, 7. doi:10.3389/fpsyt.2016.00209Parsons, S. (2016). Authenticity in Virtual Reality for assessment and intervention in autism: A conceptual review. Educational Research Review, 19, 138-157. doi:10.1016/j.edurev.2016.08.001Den Brok, W. L. J. E., & Sterkenburg, P. S. (2014). Self-controlled technologies to support skill attainment in persons with an autism spectrum disorder and/or an intellectual disability: a systematic literature review. Disability and Rehabilitation: Assistive Technology, 10(1), 1-10. doi:10.3109/17483107.2014.921248Ip, H. H. S., Wong, S. W. L., Chan, D. F. Y., Byrne, J., Li, C., Yuan, V. S. N., … Wong, J. Y. W. (2018). Enhance emotional and social adaptation skills for children with autism spectrum disorder: A virtual reality enabled approach. Computers & Education, 117, 1-15. doi:10.1016/j.compedu.2017.09.010Chen, C.-H., Lee, I.-J., & Lin, L.-Y. (2016). Augmented reality-based video-modeling storybook of nonverbal facial cues for children with autism spectrum disorder to improve their perceptions and judgments of facial expressions and emotions. Computers in Human Behavior, 55, 477-485. doi:10.1016/j.chb.2015.09.033Didehbani, N., Allen, T., Kandalaft, M., Krawczyk, D., & Chapman, S. (2016). Virtual Reality Social Cognition Training for children with high functioning autism. Computers in Human Behavior, 62, 703-711. doi:10.1016/j.chb.2016.04.033Lorenzo, G., Lledó, A., Pomares, J., & Roig, R. (2016). Design and application of an immersive virtual reality system to enhance emotional skills for children with autism spectrum disorders. Computers & Education, 98, 192-205. doi:10.1016/j.compedu.2016.03.018Wade, J., Zhang, L., Bian, D., Fan, J., Swanson, A., Weitlauf, A., … Sarkar, N. (2016). A Gaze-Contingent Adaptive Virtual Reality Driving Environment for Intervention in Individuals with Autism Spectrum Disorders. ACM Transactions on Interactive Intelligent Systems, 6(1), 1-23. doi:10.1145/2892636Ke, F., & Lee, S. (2015). Virtual reality based collaborative design by children with high-functioning autism: design-based flexibility, identity, and norm construction. Interactive Learning Environments, 24(7), 1511-1533. doi:10.1080/10494820.2015.1040421Chen, C.-H., Lee, I.-J., & Lin, L.-Y. (2015). Augmented reality-based self-facial modeling to promote the emotional expression and social skills of adolescents with autism spectrum disorders. Research in Developmental Disabilities, 36, 396-403. doi:10.1016/j.ridd.2014.10.015Cheng, Y., Huang, C.-L., & Yang, C.-S. (2015). Using a 3D Immersive Virtual Environment System to Enhance Social Understanding and Social Skills for Children With Autism Spectrum Disorders. Focus on Autism and Other Developmental Disabilities, 30(4), 222-236. doi:10.1177/1088357615583473Kim, K., Rosenthal, M. Z., Gwaltney, M., Jarrold, W., Hatt, N., McIntyre, N., … Mundy, P. (2014). A Virtual Joy-Stick Study of Emotional Responses and Social Motivation in Children with Autism Spectrum Disorder. Journal of Autism and Developmental Disorders, 45(12), 3891-3899. doi:10.1007/s10803-014-2036-7Parsons, S. (2015). Learning to work together: Designing a multi-user virtual reality game for social collaboration and perspective-taking for children with autism. International Journal of Child-Computer Interaction, 6, 28-38. doi:10.1016/j.ijcci.2015.12.002Bai, Z., Blackwell, A. F., & Coulouris, G. (2015). Using Augmented Reality to Elicit Pretend Play for Children with Autism. IEEE Transactions on Visualization and Computer Graphics, 21(5), 598-610. doi:10.1109/tvcg.2014.2385092Bekele, E., Crittendon, J., Zheng, Z., Swanson, A., Weitlauf, A., Warren, Z., & Sarkar, N. (2014). Assessing the Utility of a Virtual Environment for Enhancing Facial Affect Recognition in Adolescents with Autism. Journal of Autism and Developmental Disorders, 44(7), 1641-1650. doi:10.1007/s10803-014-2035-8Escobedo, L., Tentori, M., Quintana, E., Favela, J., & Garcia-Rosas, D. (2014). Using Augmented Reality to Help Children with Autism Stay Focused. IEEE Pervasive Computing, 13(1), 38-46. doi:10.1109/mprv.2014.19Finkelstein, S., Barnes, T., Wartell, Z., & Suma, E. A. (2013). Evaluation of the exertion and motivation factors of a virtual reality exercise game for children with autism. 2013 1st Workshop on Virtual and Augmented Assistive Technology (VAAT). doi:10.1109/vaat.2013.6786186Maskey, M., Lowry, J., Rodgers, J., McConachie, H., & Parr, J. R. (2014). Reducing Specific Phobia/Fear in Young People with Autism Spectrum Disorders (ASDs) through a Virtual Reality Environment Intervention. PLoS ONE, 9(7), e100374. doi:10.1371/journal.pone.0100374Stichter, J. P., Laffey, J., Galyen, K., & Herzog, M. (2013). iSocial: Delivering the Social Competence Intervention for Adolescents (SCI-A) in a 3D Virtual Learning Environment for Youth with High Functioning Autism. Journal of Autism and Developmental Disorders, 44(2), 417-430. doi:10.1007/s10803-013-1881-0Bekele, E., Zheng, Z., Swanson, A., Crittendon, J., Warren, Z., & Sarkar, N. (2013). Understanding How Adolescents with Autism Respond to Facial Expressions in Virtual Reality Environments. IEEE Transactions on Visualization and Computer Graphics, 19(4), 711-720. doi:10.1109/tvcg.2013.42Cai, Y., Chia, N. K. H., Thalmann, D., Kee, N. K. N., Zheng, J., & Thalmann, N. M. (2013). Design and Development of a Virtual Dolphinarium for Children With Autism. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 21(2), 208-217. doi:10.1109/tnsre.2013.2240700Ke, F., & Im, T. (2013). Virtual-Reality-Based Social Interaction Training for Children with High-Functioning Autism. The Journal of Educational Research, 106(6), 441-461. doi:10.1080/00220671.2013.832999Lorenzo, G., Pomares, J., & Lledó, A. (2013). Inclusion of immersive virtual learning environments and visual control systems to support the learning of students with Asperger syndrome. Computers & Education, 62, 88-101. doi:10.1016/j.compedu.2012.10.028Modugumudi, Y. R., Santhosh, J., & Anand, S. (2013). Efficacy of Collaborative Virtual Environment Intervention Programs in Emotion Expression of Children with Autism. Journal of Medical Imaging and Health Informatics, 3(2), 321-325. doi:10.1166/jmihi.2013.1167Wang, M., & Reid, D. (2013). Using the Virtual Reality-Cognitive Rehabilitation Approach to Improve Contextual Processing in Children with Autism. The Scientific World Journal, 2013, 1-9. doi:10.1155/2013/716890Milne, M., Luerssen, M. H., Lewis, T. W., Leibbrandt, R. E., & Powers, D. M. W. (2010). Development of a virtual agent based social tutor for children with autism spectrum disorders. The 2010 International Joint Conference on Neural Networks (IJCNN). doi:10.1109/ijcnn.2010.5596584Loomes, R., Hull, L., & Mandy, W. P. L. (2017). What Is the Male-to-Female Ratio in Autism Spectrum Disorder? A Systematic Review and Meta-Analysis. Journal of the American Academy of Child & Adolescent Psychiatry, 56(6), 466-474. doi:10.1016/j.jaac.2017.03.013Mesa-Gresa, P., Lozano, J. A., Llórens, R., Alcañiz, M., Navarro, M. D., & Noé, E. (2011). Clinical Validation of a Virtual Environment Test for Safe Street Crossing in the Assessment of Acquired Brain Injury Patients with and without Neglect. Lecture Notes in Computer Science, 44-51. doi:10.1007/978-3-642-23771-3_4Spreij, L. A., Visser-Meily, J. M. A., van Heugten, C. M., & Nijboer, T. C. W. (2014). Novel insights into the rehabilitation of memory post acquired brain injury: a systematic review. Frontiers in Human Neuroscience, 8. doi:10.3389/fnhum.2014.00993Pietrzak, E., Pullman, S., & McGuire, A. (2014). Using Virtual Reality and Videogames for Traumatic Brain Injury Rehabilitation: A Structured Literature Review. Games for Health Journal, 3(4), 202-214. doi:10.1089/g4h.2014.001

Enactivism, other minds, and mental disorders

Although enactive approaches to cognition vary in terms of their character and scope, all endorse several core claims. The first is that cognition is tied to action. The second is that cognition is composed of more than just in-the-head processes; cognitive activities are externalized via features of our embodiment and in our ecological dealings with the people and things around us. I appeal to these two enactive claims to consider a view called “direct social perception” : the idea that we can sometimes perceive features of other minds directly in the character of their embodiment and environmental interactions. I argue that if DSP is true, we can probably also perceive certain features of mental disorders as well. I draw upon the developmental psychologist Daniel Stern’s notion of “forms of vitality”—largely overlooked in these debates—to develop this idea, and I use autism as a case study. I argue further that an enactive approach to DSP can clarify some ways we play a regulative role in shaping the temporal and phenomenal character of the disorder in question, and it may therefore have practical significance for both the clinical and therapeutic encounter