Bringing action into the picture : how action influences visual awareness

This article discusses how the analysis of interactions between action and awareness allows us to better understand the mechanisms of visual awareness. We argue that action is one of several factors that influence visual awareness and we provide a number of examples. We also discuss the possible mechanisms that underlie these influences on both the cognitive and the neural levels. We propose that action affects visual awareness for the following reasons: (1) it serves as additional information in the process of evidence accumulation; (2) it restricts the number of alternatives in the decisional process; (3) it enables error detection and performance monitoring; and (4) it triggers attentional mechanisms that modify stimulus perception. We also discuss the possible neuronal mechanisms of the aforementioned effects, including feedback-dependent prefrontal cortex modulation of the activity of visual areas, error-based modulation, interhemispheric inhibition of motor cortices, and attentional modulation of visual cortex activity triggered by motor processing

Toward the autism motor signature : gesture patterns during smart tablet gameplay identify children with autism

Autism is a developmental disorder evident from infancy. Yet, its clinical identification requires expert diagnostic training. New evidence indicates disruption to motor timing and integration may underpin the disorder, providing a potential new computational marker for its early identification. In this study, we employed smart tablet computers with touch-sensitive screens and embedded inertial movement sensors to record the movement kinematics and gesture forces made by 37 children 3-6 years old with autism and 45 age- and gender-matched children developing typically. Machine learning analysis of the children’s motor patterns identified autism with up to 93% accuracy. Analysis revealed these patterns consisted of greater forces at contact and with a different distribution of forces within a gesture, and gesture kinematics were faster and larger, with more distal use of space. These data support the notion disruption to movement is core feature of autism, and demonstrate autism can be computationally assessed by fun, smart device gameplay

Does level of processing affect the transition from unconscious to conscious perception?

Abstract Recently, Windey, Gevers, and Cleeremans (2013) proposed a level of processing (LoP) hypothesis claiming that the transition from unconscious to conscious perception is influenced by the level of processing imposed by task requirements. Here, we carried out two experiments to test the LoP hypothesis. In both, participants were asked to classify briefly presented pairs of letters as same or different, based either on the letters physical features (a low-level task), or on a semantic rule (a high-level task). Stimulus awareness was measured by means of the four-point Perceptual Awareness Scale (PAS). The results showed that low or moderate stimulus visibility was reported more frequently in the low-level task than in the high-level task, suggesting that the transition from unconscious to conscious perception is more gradual in the former than in the latter. Therefore, although alternative interpretations remain possible, the results of the present study fully support the LoP hypothesis

Tablet-based gameplay identifies movement patterns related to autism spectrum disorder

Background: It has been proposed that one of the early markers of autism spectrum disorder (ASD) are abnormalities in the development of intentional movements, which can be observed from early childhood. New evidence suggests that disruption of motor timing and integration may underpin the disorder, providing a new potential marker for its identification. Objectives: In this study, we used widely available tablet devices (iPads) to identify differences in kinematics between children diagnosed with ASD and their typically developing (TD) peers. We also compared movement patterns of children diagnosed with neurodevelopmental disorders other than autism (OND) with movement patterns exhibited by ASD and TD children. We utilised tablet devices’ inertial sensors (accelerometer, gyroscope, and touchscreen to record the movements children make while playing two educational games on a tablet. Methods: Ninety-six children (aged 3-6) diagnosed with ASD, 37 diagnosed with OND, and 387 TD children took part in the study. The children were asked to play two educational games on a tablet. Each game consisted of two parts: two-minute long training and five-minute long test session. During the gameplay, we collected data from tablet’s sensors and screen. After the experimental session, 262 variables obtained by simple calculation of the raw sensor data (e.g. acceleration of the movements) were extracted and analysed using machine learning algorithms. To increase generalisation properties of the models, we reduced dimensionality to 49 most significant variables. Results: To compare movement patterns of children with ASD, OND, and TD children, we used machine learning algorithms. Each algorithm differentiated individuals within the ASD group from the other groups using 49 variables derived from the touch screen and inertial sensors. ASD - TD comparison: The algorithms classified children diagnosed with ASD from TD children with up to 93% accuracy. OND - TD comparison: The algorithms classified children diagnosed with OND from TD children with up to 95% accuracy. The results suggest that movement patterns of typically developing children are different than patterns exhibited by children diagnosed with neurodevelopmental disorders other than autism. ASD - OND comparison The algorithms classified children diagnosed with ASD from OND children with up to 93% accuracy. This result suggests that ASD is characterised by movement patterns that can be differentiated from patterns related to other neurodevelopment disorders. Conclusions: These findings support the view that children with ASD can be differentiated from TD children by movement patterns analysis. We also provide evidence suggesting that patterns characteristic of ASD children are different from patterns exhibited by children with OND. However, the latter result is not particularly strong due to the small sample of OND group. Further research is needed to provide better evidence

Potential movement biomarkers for autism in children and adolescents

Background -- While social communication deficits are the hallmark of autism spectrum disorders (ASD), motor deficits are known to be common in this population as well. Recently, members of our research team showed that kinematic markers collected by playing a tablet game may be a promising biomarker for identification of ASD as compared to a typically developing population (TD) in children ages 3-6 years old (Anzulewicz et al, 2016). To our knowledge, no one has replicated this finding in an older population. Purpose -- To replicate and extend previous findings of kinematic differences in children with ASD to an older population of children (9-14 years old). Methods -- Four TD children and 5 children with ASD (aged 9-12) played an iPad drawing game (Anzulewicz et al, 2016) that measured gesture kinematics and gesture force using inertial sensors and touch screen touch displacements. 212 features were calculated from the inertial sensor and touch screen data (ibid). A Kolmogorov-Smirnov (K-S) test was run to identify motor features distinct between ASD and TD children. Results -- K-S test identified seven significantly different features (JerkMagnitudeMax, JerkMin_y, JerkRange_y, AttitudeRange_y, RotationRMS_x, RotationStdDev_x, JerkZeroCrossing_x) between ASD and TD groups that represented differences in acceleration of finger movements and the displacement of the iPad during movements. Conclusions -- Results demonstrated inertial movement sensor parameter differences are key identifiers between 8-12 year old ASD and TD children, common to children 3-6 years old. Contact forces and the distribution of forces during coloring may serve as important identifiers of ASD irrespective of age during childhood, while other parameters may be age-dependent. Research Support NIH R01 (1R01HD079432-01A1

Smart tablet-based gameplay identification of preschool children with autism : a replication study with machine learning data analytics improvements

Background: It has been proposed that one of the early markers of autism spectrum disorder (ASD) is a disruption in intentional movement evident from early childhood. Evidence suggests disruption to motor timing and integration may underpin the disorder, providing a new potential marker for its identification. In earlier work, we demonstrated machine learning analysis of children’s movement patterns during smart tablet gameplay identified ASD with 83% sensitivity and 85% specificity (Anzulewicz, Sobota and Delafield-Butt, 2016). Objectives: In this study, we sought to test the original performance accuracy with more generalised, new data. And we sought an iterative improvement on the machine learning data analytics to simplify and further generalise the models. Overall we aimed to achieve an accessible, computational identification of ASD in young children by smart tablet gameplay. Methods: The original study of 37 children 3-6 years old with ASD and 45 children typically developing (TD) was augmented with a new dataset of 118 children with ASD and 420 TD children. In addition, 26 children 3-6 with another neurodevelopmental disorder that was not ASD was included. Feature selection was reduced by recursive feature selection and removal of low variance and high within-group correlations. New machine learning algorithms were trained on the new dataset (n=564), and these models applied to the original dataset (n=82) to test for generalisation. Results: Dimensionality was reduced from 262 kinematic and descriptive metric features of children’s gameplay patterns to 49 features. Ten repetitions of a ten-fold cross-validation procedure performed on the new dataset (n=564) identified children with ASD from their TD counterparts with 87% sensitivity and 85% specificity. Differentiation of OND from their TD counterparts was comparable, but with low confidence. Finally, we tested the models produced on the original study dataset (n=82). The model performed 83% sensitivity and 82% specificity accuracy, replicating the original finding. Conclusions: This study produced new machine learning models for the identification of ASD from TD children on a large dataset with comparable performance to the first study, and with reduced feature selection. Moreover, we replicated the findings of our previous study with these new algorithmic models, tested on those original data and without prior training on those data. We consider this strong verification of the principle of machine learning data analytics in the successful, and potentially clinically useful early identification of ASD in young children. The basis of these features on calculations of motor kinematics supports the view movement differences are a fundamental feature of ASD that may be subtle to the eye, but significantly associated computationally

The movements of children with autism can be faster or slower than their typically developing counterparts, depending on the task

Background: Atypical movement patterns in autism spectrum disorders (ASD) have been reported. Compared with typical developing (TD) children, children with ASD took more time to complete a point-to-point movement (Dowd et al., 2012), but adults with ASD performed faster horizontal arm swings than their typical counterparts (Cook et al., 2013). Incongruent kinematic results are common in the literature, which may imply that the kinematic features in ASD are task-dependent, but this is yet not well understood. Smart tablet gameplay has been proposed as a new paradigm to measure the movement features of ASD in young children (Anzulewicz et al., 2016). In this study, smart tablet games were employed to test for kinematic differences in autism, and the effect of the task. Objectives: The study aims to compute the swipe kinematics during smart tablet gameplay, and to compare these characteristic movements between ASD and TD children within different gameplay contexts. Methods: 37 ASD children (mean age: 4.5 years) and 45 age-matched TD children were recruited in the study. The children were shown two smart tablet games: "sharing" and "creativity" games. In the sharing game, the children were tasked to share the food pieces to four characters; in the creativity game, the children were tasked to select an object, trace the lines, and colour the object. Their touch trajectories on the smart tablet (iPad mini, Apple Inc.) were recorded during gameplay. The food-to-target swipes in the sharing game and the swipe gestures in the creativity game were identified using a customized MATLAB script. The travelled distance, duration, and speed of each swipe were calculated. For the sharing game, the difference between the travelled distance and the optimal distance (i.e. the straight line) was also calculated. Mann-Whitney U tests were used to determine kinematic differences between ASD and TD groups. Results: A total of 4785 food-to-target swipes were identified in the sharing game (ASD: 1585 swipes; TD: 3200 swipes) while 6178 swipes were identified in the creativity game (ASD: 2793 swipes; TD: 3385 swipes). Significant differences between ASD and TD were observed in the sharing game that ASD demonstrated slower food-to-target swipes than TD (median of 50.12 mm/s vs. 58.84 mm/s), and that they deviated from the optimal distance more than TD (median of 3.9 mm vs. 2.59 mm). There was no significant difference in the optimal distance. By contrast, ASD showed significantly faster gestures than TD (median of 81.77 mm/s vs. 60 mm/s) in the creativity game. Conclusions: The study compared the swipe kinematics between ASD and TD children in two smart tablet gameplay contexts. ASD demonstrated slower movement than TD in a goal-oriented food-to-target task, deviating more from the optimal trajectory. In contrast, ASD performed faster swipe gestures than TD in a relatively unconstrained creativity game. These data are the foundations to allow an understanding of how movement is controlled in autism within different contexts. Further, characterising movement features in ASD during smart tablet gameplay supports the development of algorithms that enable the early identification of ASD in serious game paradigms

A prospective perception-action strategy in children with autism during smart-tablet gameplay

Background: Motor differences between children with autism spectrum disorders (ASD) and those with typical development (TD) have been identified in various activities such as pointing (Torres et al., 2013) and placing (Crippa et al., 2014). Kinematic differences have also been observed in goal-oriented swipe kinematics during smart-tablet gameplay (Lu et al., 2019, 2020). General Tau Theory has been used to describe goal-oriented perception-action strategies (Lee, 2009), which proposes an intrinsic action guide generated by the nervous system coupled to the motor command to guide the physical movement. The coupling constant between the two is assumed to be set by the brain to coordinate the kinematic profile of the goal-oriented action. Here, an exploration to surface a potential difference in the tau-coupling during smart-tablet gameplay in children with ASD is presented. Objectives: To test whether or not the perception-action strategy employed by children with and without ASD differ during goal-oriented swipes in smart-tablet gameplay. Methods: Goal-oriented swipe data were extracted from a study of smart-tablet gameplay for young children (Anzulewicz et al., 2016). Only those swipes that proceeded directly from start to finish without overshooting the target were included. A total of 500 swipes were obtained from 32 children with ASD (aged 33-79 months), and 1426 swipes were obtained from 44 children with TD (aged 36-74 months). The percentage of tau-coupling in each swipe, its duration and distance, and the tau-coupling constant were determined utilising the time and x- and y-coordinates data. Results: Children with ASD demonstrated 97.90 ± 10.49 (mean ± SD) % while children with TD demonstrated 98.98 ± 7.54 % of tau-coupling movement, indicating a significantly weakening (t-test, p = 0.01) and more variable (F-test, p < 0.01) tau-coupling pattern in children with ASD. The coupling constant was 0.40 ± 0.93 for the ASD group and 0.41 ± 0.15 for the TD group. Children with ASD demonstrated a significantly wider range of the coupling constant than children with TD (F-test, p < 0.01) while the mean values were similar. Conclusions: The findings indicate that, in comparison to children with TD, children with ASD demonstrated significantly less tau-coupling with higher variability during swipes whilst engaging in smart-tablet gameplay. It should be noted that the coupling constant in ASD was significantly more variable, however, the mean value was similar to what was observed in TD. The results of the coupling constant imply that, for the overall movement, children with ASD and TD used similar strategies to perform the goal-oriented swipes while greater fluctuations were observed in ASD. These findings are consistent with previous reports indicating that individuals with ASD have difficulties in controlling goal-oriented movement efficiently with increased subsecond motor variability during the travel of the movement (Torres et al., 2013). Increased acceleration and jerk amplitudes noted in adults with ASD (Cook et al., 2013) suggests sensorimotor and timing are disrupted at the level of the brainstem integration (Delafield-Butt & Trevarthen, 2017). Therefore, disruption to efficient perception-action regulation by tau-coupling might be a critical motor disruption in ASD

Kinematics of prospective motor control in autism spectrum disorder : an exploratory multilevel modelling analysis of goal-directed finger movements during smart-tablet gameplay

Background: Disturbance in movement is widely observed in autism and differences have been measured at the level of movement kinematics. Anzulewicz et al (2016) showed that gesture patterns from smart-tablet gameplay can distinguish between children with autism (ASD) and typically developing children (TD) with high accuracy using a machine learning algorithm, but a limitation of the data-driven approach used is that distinguishing features included in the algorithm may not be grounded in theory. It has been suggested that prospective control of movement is disrupted in autism, and this may result from impairments in using sensory feedback as the movement unfolds, despite intact control of internally generated movements. Furthermore, movement kinematics variables which are influenced by task difficulty and change with motor development have been identified to indicate prospective motor control.Objectives: The objective of the analysis is to explore differences between ASD and TD children in the kinematics of prospective motor control during goal-directed finger movements to different target distances, using data collected by Anzulewicz et al (2016).Methods: Touch-screen position coordinates of 4775 goal-directed swipes made during a smart-tablet gameplay by 82 children, aged 3-5 years old, were analysed. Target distance was calculated as the length between start and end position of each swipe and five kinematic variables related to prospective motor control were calculated from time differentials of position, namely: (1) peak velocity of the full movement, (2) peak velocity of the first movement unit (1MU), (3) number of movement units (velocity peaks), (4) % time in deceleration and (5) % time to peak velocity. Multilevel modelling was used to analyse the fixed effects and interaction effect of target distance and ASD diagnosis on each kinematic outcome, including a random effect to control for correlation in the kinematic outcome for swipes by the same individual.Results: Increase in 1cm target distance led to an increase in peak velocity of the full movement, and ASD children showed a greater increase than TD (Interaction: 3%, CI: 1% to 4%, p<0.001). TD children showed a 3% reduction in peak velocity (1MU) (CI: -5% to 0%, p=0.05) and decelerate 0.41% longer (CI: 0.20% - 0.63%, p<0.001) for more distant targets, but children with ASD showed the opposite relationship (Peak velocity (1MU) - Interaction: 9%, CI: 3% to 14%, p<0.001; Deceleration - interaction: -0.54%, CI: -0.93% to -0.14%, p=0.008). ASD children reached a peak in velocity later for more distant targets (Interaction: 1.28%, CI: 0.39% to 2.16%, p=0.005), but no relationship is seen for TD children. Overall, ASD children have 31% more movement units than TD (CI: 1% to 70%, p=0.04), but a 3% smaller increase in movement units for more distant targets (CI: -5% to -1%, p=0.007).Conclusions: The kinematics of prospective control is different for children with ASD and TD, and may help to identify children with autism. These findings are consistent with the idea that individuals with ASD may differ in the use of feedback control, and internal feedforward control may be influenced differently by external constraints such as target distance