Human Attention Assessment Using A Machine Learning Approach with GAN-based Data Augmentation Technique Trained Using a Custom Dataset

Human–robot interactions require the ability of the system to determine if the user is paying attention. However, to train such systems, massive amounts of data are required. In this study, we addressed the issue of data scarcity by constructing a large dataset (containing ~120,000 photographs) for the attention detection task. Then, by using this dataset, we established a powerful baseline system. In addition, we extended the proposed system by adding an auxiliary face detection module and introducing a unique GAN-based data augmentation technique. Experimental results revealed that the proposed system yields superior performance compared to baseline models and achieves an accuracy of 88% on the test set. Finally, we created a web application for testing the proposed model in real time

Deep learning systems for estimating visual attention in robot-assisted therapy of children with autism and intellectual disability

Recent studies suggest that some children with autism prefer robots as tutors for improving their social interaction and communication abilities which are impaired due to their disorder. Indeed, research has focused on developing a very promising form of intervention named Robot-Assisted Therapy. This area of intervention poses many challenges, including the necessary flexibility and adaptability to real unconstrained therapeutic settings, which are different from the constrained lab settings where most of the technology is typically tested. Among the most common impairments of children with autism and intellectual disability is social attention, which includes difficulties in establishing the correct visual focus of attention. This article presents an investigation on the use of novel deep learning neural network architectures for automatically estimating if the child is focusing their visual attention on the robot during a therapy session, which is an indicator of their engagement. To study the application, the authors gathered data from a clinical experiment in an unconstrained setting, which provided low-resolution videos recorded by the robot camera during the child–robot interaction. Two deep learning approaches are implemented in several variants and compared with a standard algorithm for face detection to verify the feasibility of estimating the status of the child directly from the robot sensors without relying on bulky external settings, which can distress the child with autism. One of the proposed approaches demonstrated a very high accuracy and it can be used for off-line continuous assessment during the therapy or for autonomously adapting the intervention in future robots with better computational capabilities

Population Graph Cross-Network Node Classification for Autism Detection Across Sample Groups

Graph neural networks (GNN) are a powerful tool for combining imaging and non-imaging medical information for node classification tasks. Cross-network node classification extends GNN techniques to account for domain drift, allowing for node classification on an unlabeled target network. In this paper we present OTGCN, a powerful, novel approach to cross-network node classification. This approach leans on concepts from graph convolutional networks to harness insights from graph data structures while simultaneously applying strategies rooted in optimal transport to correct for the domain drift that can occur between samples from different data collection sites. This blended approach provides a practical solution for scenarios with many distinct forms of data collected across different locations and equipment. We demonstrate the effectiveness of this approach at classifying Autism Spectrum Disorder subjects using a blend of imaging and non-imaging data.Comment: To appear ICDM DMBIH workshop 202

Application of the eye tracking technology in medicine: a bibliometric analysis

Eye tracking provides a quantitative measure of eye movements during different activities. We report the results from a bibliometric analysis to investigate trends in eye tracking research applied to the study of different medical conditions. We conducted a search on the Web of Science Core Collection (WoS) database and analyzed the dataset of 2456 retrieved articles using VOSviewer and the Bibliometrix R package. The most represented area was psychiatry (503, 20.5%) followed by neuroscience (465, 18.9%) and psychology developmental (337, 13.7%). The annual scientific production growth was 11.14% and showed exponential growth with three main peaks in 2011, 2015 and 2017. Extensive collaboration networks were identified between the three countries with the highest scientific production, the USA (35.3%), the UK (9.5%) and Germany (7.3%). Based on term co-occurrence maps and analyses of sources of articles, we identified autism spectrum disorders as the most investigated condition and conducted specific analyses on 638 articles related to this topic which showed an annual scientific production growth of 16.52%. The majority of studies focused on autism used eye tracking to investigate gaze patterns with regards to stimuli related to social interaction. Our analysis highlights the widespread and increasing use of eye tracking in the study of different neurological and psychiatric conditions

Assessment of the Autism Spectrum Disorder Based on Machine Learning and Social Visual Attention: A Systematic Review

The assessment of autism spectrum disorder (ASD) is based on semi-structured procedures addressed to children and caregivers. Such methods rely on the evaluation of behavioural symptoms rather than on the objective evaluation of psychophysiological underpinnings. Advances in research provided evidence of modern procedures for the early assessment of ASD, involving both machine learning (ML) techniques and biomarkers, as eye movements (EM) towards social stimuli. This systematic review provides a comprehensive discussion of 11 papers regarding the early assessment of ASD based on ML techniques and children's social visual attention (SVA). Evidences suggest ML as a relevant technique for the early assessment of ASD, which might represent a valid biomarker-based procedure to objectively make diagnosis. Limitations and future directions are discussed

Social cognition and behavioral responses in kinematic interactions

As social beings, humans are constantly probed to infer intentions from verbal and non- verbal communication and to react according to the kinematic signals of other people. In this way, social cognition is tightly bound to our ability to perceive, predict and perform socially relevant actions. Being characterized by impairments in social interactions, in- dividuals with autism spectrum disorder (ASD) demonstrate insensitivity to predictive social stimuli as well as abnormal kinematic control both on the behavioral and the brain level. Underlining the severe consequences of impaired social interactive capabilities, autistic individuals are at high risk of social exclusion and concomitant mental health issues. Therefore, the investigation of the behavioral and brain responses to social ac- tions might yield valuable insights into the fundamental dynamics of social interactions, which could lay the foundation for clinical research and interventions in ASD. In order to provide first insights, the main goal of this thesis was to identify the non-pathological brain mechanisms in perceptual action prediction and action control within a social context. For this purpose, two functional magnetic resonance imaging (fMRI) experiments in healthy control participants were conducted: The first study of this thesis addressed the effect of observing communicative, i.e. predictive, actions on visual perception [interpersonal predictive coding (IPPC)]. By the use of point-light displays, we replicated behavioral findings of improved visual discriminability of a point-light agent after seeing a communicative as compared to an individual action of another point-light agent. Furthermore, our findings suggest a perceptual integration of social event knowledge implemented by the superior frontal gyrus (SFG) during predictive trials and a specific role of the amygdala in setting network configurations to meet the demands of the specific social context. Moving from a spectator perspective to direct involvement in a social interaction, the second study of this thesis examined the interaction of gaze processing and action control during an encounter with an anthropomorphic virtual character. The key finding of this second study comprises an increased functional coupling during high action control demands between the right temporoparietal junction (TPJ) as central gaze processing region and brain areas implicated in both action control processes and social cognition such as the inferior frontal gyri. The results of the two studies demonstrate that predictive social actions as well as direct gaze signals can modify multimodal functional integration in the brain, thereby recruiting and modulating activation in brain structures implicated in ASD. In this way, the two studies of this thesis underline the interdependence of social cognition and kinematic processes while providing a reference point for future studies on ASD

New light on neurocognitive processes linked to autism and attention deficit and hyperactivity disorder in childhood : studies of eye movements in twins

Visual attention and oculomotor response inhibition have been associated with Autism Spectrum Disorder (ASD) and Attention Deficit and Hyperactivity Disorder (ADHD) respectively. The aim of this thesis was to increase our knowledge about these cognitive functions relevant to ASD and ADHD in early infancy and childhood using eye tracking and twin modelling. Study 1 assessed the relative contribution of genetic and environmental influences to attentional networks and visual disengagement (using the gap overlap task) in a sample of twins from the general population, aged 9-14 years. It also assessed whether visual disengagement was associated with autistic traits. Gaze shift latencies across conditions were driven by shared genetic factors. Additionally, there were unique genetic influences to gaze shift latencies in the gap condition. In line with previous work, autistic traits were found to be heritable. There was no association between visual disengagement and autistic traits. Study 2 investigated the relative contribution of genetic and environmental factors to oculomotor response inhibition (using the antisaccade task) and the degree to which oculomotor response inhibition was associated with ADHD traits in the same twin sample. Oculomotor response inhibition in the form of premature anticipatory eye movements was heritable and associated to parent rated inattentive traits. This association was partially due to shared genetic factors. Study 3 investigated how visual disengagement relates to other cognitive developmental processes and behaviors, socioeconomic status and biological sex in early infancy. Gaze shift latencies in the overlap, baseline and gap conditions, of the Gap Overlap task, differed as a function of socioeconomic status and sex. No other associations between visual attention and developmental measures were observed. Thus, in summary, while these findings do not support neither a phenotypic nor a genetic link between visual disengagement and ASD, they support such association between oculomotor response inhibition and inattention (a core component of ADHD). Finally, these findings highlight the influence of sociodemographic factors on individual differences in visual attention in early infancy, thus underscoring the importance of understanding all sources of variation in attentional functions in childhood

An investigation of autonomic arousal and attentional mechanisms in children with ADHD and Autism

The present doctoral project was aimed at investigating the impact of Attention Deficit Hyperactivity Disorder (ADHD) and Autism Spectrum Disorder (ASD) on measures of physiological arousal, alerting/vigilance, attention orienting and executive functions. 106 children between 7 and 15 years of age (31 typically developing; 24 ADHD-only; 18 ASD-only; 33 ADHD&ASD) performed a battery of eye-tracking and EEG experimental paradigms, while parent-reported measures were used to evaluate the severity of symptoms of ASD, ADHD and other psychiatric conditions. Children with clinical diagnoses of ADHD and ASD showed condition-specific signs of dysregulated physiological arousal and vigilance, with ADHD more likely to be associated with difficulties in up-regulating and maintaining an optimal level of vigilance to the environment, and ASD more associated with over-reactivity to sensory information and difficulties in down-regulating autonomic arousal in line with contextual demands. We also demonstrated that executive function and cognitive control mechanisms are likely to be less effective in children with comorbid ADHD+ASD, with negative effects on performance accuracy. In the discussion of this dissertation, some suggestions for clinical practice and future research studies, besides a description of the implications of the findings on the everyday life of people with ADHD and/or ASD, are provided