Multimodal attention in a simulated driving environment - Novel approaches to the quantification of attention based on brain activity

The concept of attention is an established focus of study in neurosciences. The quantification of attention during driving can help identify situations in which the driver is not completely aware of the situation. This work deals with the implementation of a setup to simulate a driving environment that enables audiovisual tasks to be embedded into the driving task while acquiring biosignals such as electroencephalography. The main goal of this dissertation was to find a correlation between attention and brain activity as seen on the electroencephalographic activity while driving. By using the principle of phase-amplitude coupling in electroencephalographic signals, it was hypothesized that Theta-Gamma phase-amplitude coupling might correlate to multimodal attention and thus might be eligible as a biomarker of attention in tasks such as driving. Surface electroencephalography was measured simultaneously in drivers and copilots while participating in simulated driving scenarios with varying multimodal attentional demands. The phase-amplitude coupling between Theta-band phase and Gamma-band amplitude from the electroencephalograpic signal was obtained and evaluated. Results showed significant phase-amplitude coupling differences between drivers and copilots in areas related to multimodal attention (prefrontal cortex, frontal eye fields, primary motor cortex, and visual cortex). The results were confirmed by behavioral data acquired during the test (detection task). We conclude that phase-amplitude coupling does function as a biomarker for attentional demand by detecting cortical areas being activated through specific multimodal (in this case, driving) tasks. Additionally, the data acquired in the main work of this thesis was used to test an auditory stimulus reconstruction algorithm previously tested by our work group. The stimulus reconstruction allowed to obtain post-hoc additional information regarding attentional effort during driving (success of the stimulus reconstruction was significantly correlated to auditory effort) and serves as a compliment to the main results. This dissertation thus offers an insight on attentional systems in multimodal situations and the neurophysiological systems underlying attention. It develops methods to measure attention in a driving environment, both as seen using phase-amplitude coupling and by being able to single out auditory effort by reconstructing the auditory stimuli. Finally, these methods can be translated to other activities since they are both based on non-invasive electroencephalography

Neurofeedback Therapy for Enhancing Visual Attention: State-of-the-Art and Challenges

We have witnessed a rapid development of brain-computer interfaces (BCIs) linking the brain to external devices. BCIs can be utilized to treat neurological conditions and even to augment brain functions. BCIs offer a promising treatment for mental disorders, including disorders of attention. Here we review the current state of the art and challenges of attention-based BCIs, with a focus on visual attention. Attention-based BCIs utilize electroencephalograms (EEGs) or other recording techniques to generate neurofeedback, which patients use to improve their attention, a complex cognitive function. Although progress has been made in the studies of neural mechanisms of attention, extraction of attention-related neural signals needed for BCI operations is a difficult problem. To attain good BCI performance, it is important to select the features of neural activity that represent attentional signals. BCI decoding of attention-related activity may be hindered by the presence of different neural signals. Therefore, BCI accuracy can be improved by signal processing algorithms that dissociate signals of interest from irrelevant activities. Notwithstanding recent progress, optimal processing of attentional neural signals remains a fundamental challenge for the development of efficient therapies for disorders of attention

High contextual sensitivity of metaphorical expressions and gesture blending: A video event-related potential design

Human communication in a natural context implies the dynamic coordination of contextual clues, paralinguistic information and literal as well as figurative language use. In the present study we constructed a paradigm with four types of video clips: literal and metaphorical expressions accompanied by congruent and incongruent gesture actions. Participants were instructed to classify the gesture accompanying the expression as congruent or incongruent pressing two different keys while electrophysiological activity was being recorded. We compared behavioral measures and event related potential (ERP) differences triggered by the gesture stroke onset. Accuracy data showed that incongruent metaphorical expressions were more difficult to classify. Reaction times were modulated by incongruent gestures, by metaphorical expressions and for a gesture–expression interaction. No behavioral differences were found between the literal and metaphorical expressions when gesture was congruent. N400-like and LPC-like (late positive complex)components from metaphorical expressions produced greater negativity. The N400-like modulation of metaphorical expressions showed a greater difference between congruent and incongruent categories over the left anterior region, compared with the literal expressions. More importantly, the literal congruent as well as the metaphorical congruent categories did not show any difference. Accuracy, reaction times and ERPs provide convergent support for a greater contextual sensitivity of the metaphorical expressions.Fil: Ibañez, Agustin Mariano. Universidad Favaloro; Argentina. Instituto de Neurología Cognitiva; Argentina. Universidad Diego Portales; Chile. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Toro, Pablo. Universidad Católica de Chile; Chile. Pontificia Universidad Católica de Chile; ChileFil: Cornejo, Carlos. Pontificia Universidad Católica de Chile; Chile. Universidad Católica de Chile; ChileFil: Hurquina, Hugo. Instituto de Neurología Cognitiva; Argentina. Universidad Favaloro; ArgentinaFil: Manes, Facundo Francisco. Universidad Favaloro; Argentina. Instituto de Neurología Cognitiva; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Weisbrod, Matthias. Universität Heidelberg; AlemaniaFil: Schröder, Johannes. Universität Heidelberg; Alemani

Influence of Auditory Cues on the Neuronal Response to Naturalistic Visual Stimuli in a Virtual Reality Setting

Virtual reality environments offer great opportunities to study the performance of brain-computer interfaces (BCIs) in real-world contexts. As real-world stimuli are typically multimodal, their neuronal integration elicits complex response patterns. To investigate the effect of additional auditory cues on the processing of visual information, we used virtual reality to mimic safety-related events in an industrial environment while we concomitantly recorded electroencephalography (EEG) signals. We simulated a box traveling on a conveyor belt system where two types of stimuli – an exploding and a burning box – interrupt regular operation. The recordings from 16 subjects were divided into two subsets, a visual-only and an audio-visual experiment. In the visual-only experiment, the response patterns for both stimuli elicited a similar pattern – a visual evoked potential (VEP) followed by an event-related potential (ERP) over the occipital-parietal lobe. Moreover, we found the perceived severity of the event to be reflected in the signal amplitude. Interestingly, the additional auditory cues had a twofold effect on the previous findings: The P1 component was significantly suppressed in the case of the exploding box stimulus, whereas the N2c showed an enhancement for the burning box stimulus. This result highlights the impact of multisensory integration on the performance of realistic BCI applications. Indeed, we observed alterations in the offline classification accuracy for a detection task based on a mixed feature extraction (variance, power spectral density, and discrete wavelet transform) and a support vector machine classifier. In the case of the explosion, the accuracy slightly decreased by –1.64% p. in an audio-visual experiment compared to the visual-only. Contrarily, the classification accuracy for the burning box increased by 5.58% p. when additional auditory cues were present. Hence, we conclude, that especially in challenging detection tasks, it is favorable to consider the potential of multisensory integration when BCIs are supposed to operate under (multimodal) real-world conditions

Semantic radical consistency and character transparency effects in Chinese: an ERP study

BACKGROUND: This event-related potential (ERP) study aims to investigate the representation and temporal dynamics of Chinese orthography-to-semantics mappings by simultaneously manipulating character transparency and semantic radical consistency. Character components, referred to as radicals, make up the building blocks used dur...postprin

Social and Affective Neuroscience of Everyday Human Interaction

This Open Access book presents the current state of the art knowledge on social and affective neuroscience based on empirical findings. This volume is divided into several sections first guiding the reader through important theoretical topics within affective neuroscience, social neuroscience and moral emotions, and clinical neuroscience. Each chapter addresses everyday social interactions and various aspects of social interactions from a different angle taking the reader on a diverse journey. The last section of the book is of methodological nature. Basic information is presented for the reader to learn about common methodologies used in neuroscience alongside advanced input to deepen the understanding and usability of these methods in social and affective neuroscience for more experienced readers