Shifts of Attention During Spatial Language Comprehension: A Computational Investigation

Kluth T, Burigo M, Knoeferle P. Shifts of Attention During Spatial Language Comprehension: A Computational Investigation. In: van den Herik J, Filipe J, eds. Proceedings of the 8th International Conference on Agents and Artificial Intelligence (ICAART 2016). Vol 2. Rome, Italy: SCITEPRESS – Science and Technology Publications, Lda.; 2016: 213-222.Regier and Carlson (2001) have investigated the processing of spatial prepositions and developed a cognitive model that formalizes how spatial prepositions are evaluated against depicted spatial relations between objects. In their Attentional Vector Sum (AVS) model, a population of vectors is weighted with visual attention, rooted at the reference object and pointing to the located object. The deviation of the vector sum from a reference direction is then used to evaluate the goodness-of-fit of the spatial preposition. Crucially, the AVS model assumes a shift of attention from the reference object to the located object. The direction of this shift has been challenged by recent psycholinguistic and neuroscientific findings. We propose a modified version of the AVS model (the rAVS model) that integrates these findings. In the rAVS model, attention shifts from the located object to the reference object in contrast to the attentional shift from the reference object to the located object implemented in the AVS model. Our model simulations show that the rAVS model accounts for both the data that inspired the AVS model and the most recent findings

Multimodal Comprehension of Language and Graphics: Graphs with and without annotations

An experimental investigation into interaction between language and information graphics in multimodal documents served as the basis for this study. More specifically, our purpose was to investigate the role of linguistic annotations in graph-text documents. Participants were presented with three newspaper articles in the following conditions: one text-only, one text plus non-annotated graph, and one text plus annotated graph. Results of the experiment showed that, on one hand, annotations play a bridging role for integration of information contributed by different representational modalities. On the other hand, linguistic annotations have negative effects on recall, possibly due to attention divided by the different parts of a document

Situated Sentence Processing: The Coordinated Interplay Account and a Neurobehavioral Model

Crocker MW, Knoeferle P, Mayberry M. Situated Sentence Processing: The Coordinated Interplay Account and a Neurobehavioral Model. Brain and Language. 2010;112(3):189-201

Value and efficacy of transcranial direct current stimulation in the rehabilitation of neurocognitive disorders: A critical review since 2000.

open3siNon-invasive brain stimulation techniques, including transcranial direct current stimulation (t-DCS) have been used in the rehabilitation of cognitive function in a spectrum of neurological disorders. The present review outlines methodological communalities and differences of t-DCS procedures in neurocognitive rehabilitation. We consider the efficacy of tDCS for the management of specific cognitive deficits in four main neurological disorders by providing a critical analysis of recent studies that have used t-DCS to improve cognition in patients with Parkinson’s Disease, Alzheimer’s Disease, Hemi-spatial Neglect and Aphasia. The evidence from this innovative approach to cognitive rehabilitation suggests that tDCS can influence cognition. However, the results show a high variability between studies both on the methodological approach adopted and the cognitive functions aspects. The review also focuses both on methodological issues such as technical aspects of the stimulation ( electrodes position and dimension; current intensity; duration of protocol) and on the inclusion of appropriate assessment tools for cognition. A further aspect considered is the best timing to administer tDCS: before, during after cognitive rehabilitation. We conclude that more studies with shared methodology are needed to have a better understanding of the efficacy of tDCS as a new tool for rehabilitation of cognitive disorders in a range of neurological disordersopenCappon, D; Jahanshahi, M; Bisiacchi, PCappon, Davide; Jahanshahi, M; Bisiacchi, Patrizi

Cross-modal extinction in a boy with severely autistic behaviour and high verbal intelligence

Anecdotal reports from individuals with autism suggest a loss of awareness to stimuli from one modality in the presence of stimuli from another. Here we document such a case in a detailed study of T.M., a 13-year-old boy with autism in whom significant autistic behaviors are combined with an uneven IQ profile of superior verbal and low performance abilities. Although T.M.'s speech is often unintelligible and his behavior is dominated by motor stereotypies and impulsivity, he can communicate by typing or pointing independently within a letter board. A series of experiments using simple and highly salient visual, auditory, and tactile stimuli demonstrated a hierarchy of cross-modal extinction, in which auditory information extinguished other modalities at various levels of processing. T.M. also showed deficits in shifting and sustaining attention. These results provide evidence for mono-channel perception in autism and suggest a general pattern of winner-takes-all processing in which a stronger stimulus-d riven representation dominates behavior, extinguishing weaker representations

Speed-accuracy strategy regulations in prefrontal tumor patients

The ability to flexibly switch between fast and accurate decisions is crucial in everyday life. Recent neuroimaging evidence suggested that left lateral prefrontal cortex plays a role in switching from a quick response strategy to an accurate one. However, the causal role of the left prefrontal cortex in this particular, non-verbal, strategy switch has never been demonstrated. To fill this gap, we administered a perceptual decision-making task to neuro-oncological prefrontal patients, in which the requirement to be quick or accurate changed randomly on a trial-by-trial basis. To directly assess hemispheric asymmetries in speed-accuracy regulation, patients were tested a few days before and a few days after surgical excision of a brain tumor involving either the left (N=13) or the right (N=12) lateral frontal brain region. A group of age- and education-matched healthy controls was also recruited. To gain more insight on the component processes implied in the task, performance data (accuracy and speed) were not only analyzed separately but also submitted to a diffusion model analysis. The main findings indicated that the left prefrontal patients were impaired in appropriately adopting stricter response criteria in speed-to-accuracy switching trials with respect to healthy controls and right prefrontal patients, who were not impaired in this condition. This study demonstrates that the prefrontal cortex in the left hemisphere is necessary for flexible behavioral regulations, in particular when setting stricter response criteria is required in order to successfully switch from a speedy strategy to an accurate one

Computational modelling of neural mechanisms underlying natural speech perception

Humans are highly skilled at the analysis of complex auditory scenes. In particular, the human auditory system is characterized by incredible robustness to noise and can nearly effortlessly isolate the voice of a specific talker from even the busiest of mixtures. However, neural mechanisms underlying these remarkable properties remain poorly understood. This is mainly due to the inherent complexity of speech signals and multi-stage, intricate processing performed in the human auditory system. Understanding these neural mechanisms underlying speech perception is of interest for clinical practice, brain-computer interfacing and automatic speech processing systems. In this thesis, we developed computational models characterizing neural speech processing across different stages of the human auditory pathways. In particular, we studied the active role of slow cortical oscillations in speech-in-noise comprehension through a spiking neural network model for encoding spoken sentences. The neural dynamics of the model during noisy speech encoding reflected speech comprehension of young, normal-hearing adults. The proposed theoretical model was validated by predicting the effects of non-invasive brain stimulation on speech comprehension in an experimental study involving a cohort of volunteers. Moreover, we developed a modelling framework for detecting the early, high-frequency neural response to the uninterrupted speech in non-invasive neural recordings. We applied the method to investigate top-down modulation of this response by the listener's selective attention and linguistic properties of different words from a spoken narrative. We found that in both cases, the detected responses of predominantly subcortical origin were significantly modulated, which supports the functional role of feedback, between higher- and lower levels stages of the auditory pathways, in speech perception. The proposed computational models shed light on some of the poorly understood neural mechanisms underlying speech perception. The developed methods can be readily employed in future studies involving a range of experimental paradigms beyond these considered in this thesis.Open Acces

Different Methods of Embodied Cognition in Pedagogy and its Effectiveness in Student Learning

The Mathematical Ideas Analysis hypothesizes that abstract mathematical reasoning is unconsciously organized and integrated with sensory-motor experience. Basic research testing movement, language, and perception during math problem solving supports this hypothesis. Applied research primarily measures students’ performance on math tests after they engage in analogous sensory-motor tasks, but findings show mixed results. Sensory-motor tasks are dependent on several moderators (e.g., instructional guidance, developmental stage) known to help students learn, and studies vary in how each moderator is implemented. There is little research on the effectiveness of sensory-motor tasks without these moderators. This study compares different approaches to working with an interactive application designed to emulate how people intrinsically solve algebraic equations. A total of 130 participants (84 females, 54 males) were drawn from a pool of Introductory Psychology students attending San Jose State University. Participants were placed in three different learning environments, and their performance was measured by comparing improvement between a pre-test and a post-test. We found no difference between participants who worked alone with the application, were instructed by the experimenter while using the application, or who instructed the experimenter on how to solve equations using the application. Further research is needed to examine how and whether analogous sensory-motor interfaces are a useful learning tool, and if so, what circumstances are ideal for sensory-motor interfaces to be used

Creativity and the Brain

Neurocognitive approach to higher cognitive functions that bridges the gap between psychological and neural level of description is introduced. Relevant facts about the brain, working memory and representation of symbols in the brain are summarized. Putative brain processes responsible for problem solving, intuition, skill learning and automatization are described. The role of non-dominant brain hemisphere in solving problems requiring insight is conjectured. Two factors seem to be essential for creativity: imagination constrained by experience, and filtering that selects most interesting solutions. Experiments with paired words association are analyzed in details and evidence for stochastic resonance effects is found. Brain activity in the process of invention of novel words is proposed as the simplest way to understand creativity using experimental and computational means. Perspectives on computational models of creativity are discussed