Emotions Recognition in people with Autism using Facial Expressions and Machine Learning Techniques: Survey

في الآونة الأخيرة ، اهتمت الكثير من الدراسات بالتعرف على المشاعر واكتشافها لدى الأشخاص المصابين بالتوحد. الهدف الرئيسي من هذه الورقة هو مسح الدراسات المختلفة التي تتعلق بالحالة العاطفية للأشخاص المصابين بالتوحد. يتضمن الاستطلاع جزأين ، يركز الجزء الأول على الدراسات التي استخدمت تعابير الوجه للتعرف على المشاعر واكتشافها. حيث تعتبر تعبيرات الوجه من التقنيات العاطفية المهمة التي تستخدم للتعبير عن أنماط مختلفة من المشاعر. ركزت الأجزاء الثانية من هذه الدراسة على الأساليب التقنية المختلفة مثل التعلم الآلي والتعلم العميق والخوارزميات الأخرى التي تستخدم لتحليل وتحديد سلوكيات الوجه للأشخاص المصابين بالتوحد. للعثور على الحل الأمثل ، يتم من خلال التحقيق في مقارنة أنظمة الكشف عن المشاعر الحالية في هذه الورقة.Recently, a lot of studies have been interested in recognizing and detection of emotions in people with autism.  The main goal of this paper is to survey different studies which have been concerned emotional state of people with autism.  The survey includes two parts, first one focused on studies which use facial expressions to recognize and detect emotions. As facial expressions are considered the affective and important techniques which is used to express different patterns of emotions.  Second parts of this study, focuses on different technical methods like machine learning, deep learning and other algorithms that are employed to analyze and determine the facial behaviors of people with autism. To find the optimal solution, a comparison of current emotion-detecting systems is investigated in this paper

Using Virtual Reality to explore individual differences in perception due to neurodiversity

The aim of this thesis is to contribute to our understanding of individual differences in visual perception, specifically in autistic and ADHD traits as well as associated diagnosed groups; and explore the use of Virtual Reality (VR) environments to enhance communication and creative expression for these individuals. The thesis begins by introducing autism and ADHD, emphasizing the perceptual differences associated with the conditions. It also highlights the importance of a person-centric approach in research and introduces the use of drawing as a research method, as it allows the capture of subjective experiences. We do so to better understand how VR as a research platform can help us study individual differences. Previous research neglects perceptual and cognitive aspects of neurodivergence in VR research and lacks clear systemized, theoretical and methodological standards; as we demonstrate in three consecutive literature reviews on VR applications in autism research. Using mixed methods and arts-based research we provide a more comprehensive understanding of the topic. Feasibility studies investigate perceptual differences in local and global processing, using the Rey Osterrieth Complex Figure (ROCF) task and free drawing. We demonstrate a link between attention-related traits and performance on visual tasks, such as ROCF. Moreover, we introduce novel methodology for evaluating two-dimensional and three-dimensional drawings and triangulating this information with qualitative thematic analysis. Furthermore, our free drawing task reveals that simplistic immersive virtual environments are viewed favorably by autistic individuals, and participants often share their thought processes spontaneously, potentially suggesting reduction in the power imbalance between the researchers and the participants. The significance of this study is that we provide evidence for the feasibility of a new methodological approach (drawing in VR) to understand perceptual differences associated with neurodiversity

An investigation as to how a computerised multimedia intervention could be of use for practitioners supporting learners with Autism Spectrum Disorder (ASD)

This practice-based action research investigation seeks to make a valuable, original and academic contribution to knowledge in the computing, language, communication and educational fields. The aim was to establish the therapeutic (language and communication skills) and educational (literacy and numeracy skills) use of individual tailored computer games for practitioners supporting learners (end-users) with Autism Spectrum Disorder (ASD). This was achieved through a continuous collaboration of cohorts of computing undergraduate students and academics (the development team) carrying out an assignment for a module designed and successfully led by this PhD student (the researcher). The researcher continually collaborated with practitioners (users – teaching staff and speech and language therapists in schools) of learners with ASD over many years. The researcher developed a Computerised Multimedia Therapeutic/Educational Intervention (CMT/EI) process, which used an iterative holistic Design-For-One approach for developing individual computer games. An action research methodology was adopted using methodological triangulation ‘quantitative’ and ‘qualitative’ data collection methods. This was to ascertain as to how tailor-made computerised multimedia games developed, could be evaluated by the users as being of therapeutic/educational use for their learners (end-users) with ASD. The researcher originated profiles to establish the diversity of each learner’s spectrum of therapeutic/educational autistic needs, preferences, capabilities, likes, dislikes and interests. The researcher orchestrated, collaborated and supervised the whole process from individual profiles completed by the practitioners, through to the profiles used as a baseline, by the development team, and to the designing, developing and evaluating iterative customised personalised computer games. Four hundred and sixty-four learners with ASD (end-users) and forty-nine practitioners (users) from nine educational establishments across the UK participated in this investigation. Two stages were carried out in an initial application procedure (with one school) and prototype procedure (with a further six schools and 2 educational establishments). Stage I - Planning, collection, organisation, Design-For-One approach and development. Stage II - Testing, Evaluation, Monitoring, Reflection and Maintenance. Optimistic ‘quantitative’ and ‘qualitative’ evidence emerged (using content analysis) from the implementation of games in the classroom and the practitioner’s therapeutic and educational evaluation of storyboards and games. The documented positive findings led to a conclusion that personalised games which had been developed over a ten-year period, showed to be of therapeutic/educational use to practitioners and their learners with ASD

Real-time 3D Graphic Augmentation of Therapeutic Music Sessions for People on the Autism Spectrum

This thesis looks at the requirements analysis, design, development and evaluation of an application, CymaSense, as a means of improving the communicative behaviours of autistic participants through therapeutic music sessions, via the addition of a visual modality. Autism spectrum condition (ASC) is a lifelong neurodevelopmental disorder that can affect people in a number of ways, commonly through difficulties in communication. Interactive audio-visual feedback can be an effective way to enhance music therapy for people on the autism spectrum. A multi-sensory approach encourages musical engagement within clients, increasing levels of communication and social interaction beyond the sessions.Cymatics describes a resultant visualised geometry of vibration through a variety of mediums, typically through salt on a brass plate or via water. The research reported in this thesis focuses on how an interactive audio-visual application, based on Cymatics, might improve communication for people on the autism spectrum.A requirements analysis was conducted through interviews with four therapeutic music practitioners, aimed at identifying working practices with autistic clients. CymaSense was designed for autistic users in exploring effective audio-visual feedback, and to develop meaningful cross-modal mappings of musical practitioner-client communication. CymaSense mappings were tested by 17 high functioning autistic participants, and by 30 neurotypical participants. The application was then trialled as a multimodal intervention for eight participants with autism, over a 12-week series of therapeutic music sessions. The study captured the experiences of the users and identified behavioural changes as a result, including information on how CymaSense could be developed further. This dissertation contributes evidence that multimodal applications can be used within therapeutic music sessions as a tool to increase communicative behaviours for autistic participants

Spotting lies and reading minds: development of mentalizing and deception in autistic and non-autistic individuals

Deception is ever-present in day-to-day life. One cognitive process underlying deception, which has been observed to evolve throughout development, is mentalizing i.e. the ability to attribute mental-states to others. Autistic individuals have been found to struggle with mentalizing even in adulthood, so it is possible that they show difficulties in detecting deception as well. The main aims of this PhD were to investigate how mentalizing and deception develop in autistic and non-autistic individuals from pre-adolescence to early adulthood, and to investigate other factors that may affect deception judgement, specifically intergroup bias. In my first study, I collected deception stimuli for two novel deception detection tasks, and investigated if mentalizing ability and autistic traits in a non-autistic sample were related to how successful one is at deceiving. I found that, contrary to expectations, deception production success did not correlate with either mentalizing or autistic traits. For my second study, I tested 11-30 years old autistic and non-autistic participants, using a well-established detection paradigm as well as two novel deception detection tasks, and found that autistic individuals were weaker at detecting deception than non-autistic individuals. While both mentalizing and deception detection abilities improved with age in non-autistic individuals, neither were affected by age in autistic individuals. Furthermore, deception detection was found to predict peer-victimization, and through peer-victimization effect psychological distress. For my final study, I investigated neurotype-based intergroup bias in the context of deception and found that, instead of better deception detection for the in-group (vs out-group) that was expected, both autistic and non-autistic adults were better at detecting deception from other autistic adults. I discuss the theoretical implications of these finding in terms of our understanding of the cognitive underpinnings of deception, the implications this has for autistic individuals’ quality of life, and future avenues for deception and autism research

The role of autistic traits in the perception of emotion from faces and voices: a behavioural and fMRI investigation

This thesis combined behavioural and fMRI approaches in the study of the role of autistic traits in the perception of emotion from faces and voices, addressing research questions concerning: behavioural recognition of a full range of six basic emotions across multiple domains (face, voice, and face-voice); neural correlates during the processing of a wide range of emotional expressions from the face, the voice and the combination of both; neural circuity in responding to an incongruence effect (incongruence vs. congruence). The behavioural study investigated the effects of autistic traits as quantified by the Autism- Spectrum Quotient (AQ) on emotional processing in forms of unimodal (faces, voices) and crossmodal (emotionally congruent face-voice expressions) presentations. In addition, by taking into account the degree of anxiety, the role of co-morbid anxiety on emotion recognition in autistic traits was also explored. Compared to an age and gender-matched group of individuals with low levels of autistic traits (LAQ), a trend of no general deficit was found in individuals with high levels of autistic traits (HAQ) in recognizing emotions presented in faces and voice, regardless of their co-morbid anxiety. However, co-morbid anxiety did moderate the relationship between autistic traits and the recognition of emotions (e.g., fear, surprise, and anger), and this effect tended to be different for the two groups. Specifically, with greater anxiety, individuals with HAQ were found to show less probility of correct response in recognizing the emotion of fear. In contrast, individuals with LAQ showed greater probability of correct response in recognizing fear expressions. For response time, anxiety symptoms tended to be significantly associated with greater response latency in the HAQ group but less response latency in the LAQ group in the recognition of emotional expressions, negative emotions in particular (e.g., anger, fear, and sadness); and this effect of anxiety was not restricted to specific modalities. Despite the absence of finding a general emotion recognition deficit in individuals with considerable autistic traits compared to those with low levels of autistic traits, it did not necessarily mean that these two groups shared same neural network when processing emotions. Therefore, it was useful to explore the neural correlates engaged in processing of emotional expressions in individuals with high levels of autistic traits. Results of this investigation tended to suggest a hypo activation of brain areas dedicated to multimodal integration, particularly for displays showing happiness and disgust. However, both the HAQ group and LAQ group showed similar patterns of brain response (mainly in temporal regions) in response to face-voice combination. In response to emotional stimuli in single modality, the HAQ group activated a number of frontal and temporal regions (e.g., STG, MFG, IFG); these differences may suggested a more effortful and less automatic processing in individual with HAQ. In everyday life, emotional information is often conveyed by both the face and voice. Consequently, concurrently presented information by one source can alter the way that information from the other source is perceived and leads to emotional incongruence if information from the two sources was incongruent. Using fMRI, the present work also examined the neural circuity involved in responding to an incongruence effect (incongruence vs. congruence) from face-voice pairs in a group of individuals with considerable autistic traits. In addition, the differences in brain responses for emotional incongruity between explicit instructions to attend to facial expression and explicit instructions to attend to tone of voice in autistic traits was also explored. It was found that there was no significant incongruence effect between groups, given that individuals with a high level of autistic traits are able to recruit more normative neural networks for processing incongruence as individuals with a low level of autistic traits, regardless of instructions. Though no between group differences, individuals with HAQ showed negative activation in regions involved in the default- mode network. However, taken into account changes of instructions, a stronger incongruence effect was more likely to be occurred in the voice-attend condition for individuals with HAQ while in the face-attend condition for individuals with LAQ

Integration of Technology Within Intervention Strategies for Students With High Functioning Autism: A Phenomenological Approach to Analyzing Educators’ Viewpoints

There is a phenomenon that exists within the Maryland State Public School System regarding technology integration within intervention strategies for students with high functioning autism (HFA). Educators have attested that there is minimally available technology for consistent use when working with their students during intervention strategies and services. Thus, when stakeholders understand the actual experiences of the professionals that work with students that have HFA on a daily basis, positive reform may occur at the immediate level by administrators within school buildings. The purpose of this study was to examine how general and special educators experienced technology use during interventions that they provided to their students with HFA. There were two main research questions: How do general and special educators describe their experiences using technology during interventions for students with HFA? What factors are IEP team committee members considering when they decide to include or refrain from adding technology accommodations within an IEP for students with HFA? The instrumentation utilized in this study was a set of open-ended questions conducted in an interview format. After careful analysis of the data collected, six main themes were detected connected to the conceptual framework of educational equity, persuasive technology, and theory of mind. The necessity of serious funding reform for technology within this particular county are the implications for future practices in the Maryland State public school system. Provision of technology including electronic devices, adequate professional development, and increased funding will equalize educational access for disabled students with HFA

2014 Annual Research Symposium Abstract Book

2014 annual volume of abstracts for science research projects conducted by students at Trinity College

Teaching Students with Autism in an Inclusive Environment: A Case Study

The purpose of this case study was to understand the perspectives and experiences of general education teachers who teach high school students with autism spectrum disorder in inclusive education with respect to challenges teachers face, strategies and interventions they use to manage the classroom, and relationships they build to create an environment where students with autism spectrum disorder can learn and grow. Participants were identified via purposeful criterion sampling procedures, based on general education teachers in inclusive education. The setting for this study was an inclusive classroom in a public senior high school in North Carolina, referred to herein as Johnathon Brandon High School. Premack and Woodruff’s theory of mind was used to guide this study and provide a clear understanding of the inability of youth with autism spectrum disorder to decipher facial expressions and body language, the pitch or tone of a person’s voice, and the deeper meaning in conversations. The central research question focused on general education high school teachers’ descriptions of the ways inclusive education impacted high school students with autism spectrum disorder. The sub-questions for this study focused on the challenges, strategies and intervention techniques, and general education teachers’ efforts establishing a community based on collaboration. Data were collected and analyzed from an online survey, semi-structured interviews of 12 participants, and focus group discussion. Results revealed that participants desired more training to have a better understanding of autism spectrum disorder and the inclusive environment. Additionally, participants resolved that collaborative relationships established in inclusive education will help create an environment where students with autism spectrum disorder can learn and grow