Applications of brain imaging methods in driving behaviour research

Applications of neuroimaging methods have substantially contributed to the scientific understanding of human factors during driving by providing a deeper insight into the neuro-cognitive aspects of driver brain. This has been achieved by conducting simulated (and occasionally, field) driving experiments while collecting driver brain signals of certain types. Here, this sector of studies is comprehensively reviewed at both macro and micro scales. Different themes of neuroimaging driving behaviour research are identified and the findings within each theme are synthesised. The surveyed literature has reported on applications of four major brain imaging methods. These include Functional Magnetic Resonance Imaging (fMRI), Electroencephalography (EEG), Functional Near-Infrared Spectroscopy (fNIRS) and Magnetoencephalography (MEG), with the first two being the most common methods in this domain. While collecting driver fMRI signal has been particularly instrumental in studying neural correlates of intoxicated driving (e.g. alcohol or cannabis) or distracted driving, the EEG method has been predominantly utilised in relation to the efforts aiming at development of automatic fatigue/drowsiness detection systems, a topic to which the literature on neuro-ergonomics of driving particularly has shown a spike of interest within the last few years. The survey also reveals that topics such as driver brain activity in semi-automated settings or the brain activity of drivers with brain injuries or chronic neurological conditions have by contrast been investigated to a very limited extent. Further, potential topics in relation to driving behaviour are identified that could benefit from the adoption of neuroimaging methods in future studies

Proceedings of the 3rd International Mobile Brain/Body Imaging Conference : Berlin, July 12th to July 14th 2018

The 3rd International Mobile Brain/Body Imaging (MoBI) conference in Berlin 2018 brought together researchers from various disciplines interested in understanding the human brain in its natural environment and during active behavior. MoBI is a new imaging modality, employing mobile brain imaging methods like the electroencephalogram (EEG) or near infrared spectroscopy (NIRS) synchronized to motion capture and other data streams to investigate brain activity while participants actively move in and interact with their environment. Mobile Brain / Body Imaging allows to investigate brain dynamics accompanying more natural cognitive and affective processes as it allows the human to interact with the environment without restriction regarding physical movement. Overcoming the movement restrictions of established imaging modalities like functional magnetic resonance tomography (MRI), MoBI can provide new insights into the human brain function in mobile participants. This imaging approach will lead to new insights into the brain functions underlying active behavior and the impact of behavior on brain dynamics and vice versa, it can be used for the development of more robust human-machine interfaces as well as state assessment in mobile humans.DFG, GR2627/10-1, 3rd International MoBI Conference 201

Valutazione degli stati mentali attraverso l'utilizzo di interfacce cervello-computer passive

The monitoring of cognitive functions aims at gaining information about the current cognitive state of the user by decoding brain signals. In recent years, this approach allowed to acquire valuable information about the cognitive aspects regarding the interaction of humans with external world. From this consideration, researchers started to consider passive application of brain–computer interface (BCI) in order to provide a novel input modality for technical systems solely based on brain activity. The objective of this thesis is to demonstrate how the passive Brain Computer Interfaces (BCIs) applications can be used to assess the mental states of the users, in order to improve the human machine interaction. Two main studies has been proposed. The first one allows to investigate whatever the Event Related Potentials (ERPs) morphological variations can be used to predict the users’ mental states (e.g. attentional resources, mental workload) during different reactive BCI tasks (e.g. P300-based BCIs), and if these information can predict the subjects’ performance in performing the tasks. In the second study, a passive BCI system able to online estimate the mental workload of the user by relying on the combination of the EEG and the ECG biosignals has been proposed. The latter study has been performed by simulating an operative scenario, in which the occurrence of errors or lack of performance could have significant consequences. The results showed that the proposed system is able to estimate online the mental workload of the subjects discriminating three different difficulty level of the tasks ensuring a high reliability.La valutazione delle funzioni cognitive ha l’obbiettivo di ottenere informazioni sullo stato mentale attuale dell'utente, attraverso la decodifica dei segnali cerebrali. Negli ultimi anni, questo approccio ha consentito di indagare informazioni preziose sugli aspetti cognitivi riguardanti l'interazione tra l’uomo ed il mondo esterno. In base a queste considerazioni, recentemente si è considerata in letteratura la possibilità di utilizzare le interfacce cervello computer passive (BCI passivi) per interagire con dispositivi esterni, sfruttando l’attività spontanea dell’utente. L'obiettivo di questa tesi è quello di dimostrare come le interfacce cervello computer passive possano essere utilizzate per valutare lo stato mentale dell’utente, al fine di migliorare l'interazione uomo-macchina. Sono stati presentati due studi principali. Il primo ha l’obbiettivo di investigare le variazioni morfologiche dei potenziali evento correlati (ERP), al fine di associarle agli stati mentali dell’utente (es. attenzione, carico di lavoro mentale) durante l’utilizzo di BCI reattive, e come predittori delle performance raggiunte dai soggetti. Nel secondo studio è stato sviluppato e validato un sistema BCI passivo in grado di stimare il carico di lavoro mentale dell'utente durante task operative, attraverso la combinazione del segnale elettroencefalografico (EEG) ed elettrocardiografico (ECG). Quest'ultimo studio è stato effettuato simulando uno scenario operativo, in cui il verificarsi di errori da parte dell’operatore o il calo di prestazioni poteva avere conseguenze importanti. I risultati hanno mostrato che il sistema proposto è in grado di discriminare il carico di lavoro mentale percepito dall’utente su tre livelli di difficoltà, garantendo un’elevata affidabilità

Auditory cues for attention management

An exhaustible supply of mental resources necessitate that we are selective for what we attend to. Attention prioritizes what ought to be processed and what ignored, allocating valuable resources to selected information at the cost of unattended information elsewhere. For this purpose it is necessary to know the conditions that help the brain decide when attention should be paid, where to and to what information. The question that is central to this dissertation is how auditory cues can support the management of limited attentional resources based on auditory characteristics. Auditory cues can (1) increase the overall alertness, (2) orient attention to unattended information, or (3) manage attentional resources by informing of an upcoming task-switch and, therefore, indicate when to pay attention to which task. The first study of this dissertation investigated whether different population groups might process auditory cues differently, thus resulting in different levels of alertness (1). Study two examined more specifically whether the type of auditory cue (verbal command or auditory icon) used as in-vehicle notifications can influence the level of alertness (1). Studies three and four investigated the use of a special auditory cue characteristic, the looming intensity profile, for directing attention to regions of interest (2). Here, attention orienting to peripheral events was tested within a dual-task paradigm which required attention shifts between the two tasks (3). Throughout the studies, I show that electroencephalography (EEG) is an indispensable tool for evaluating auditory cues and their influence on crossmodal attention. By using EEG measurements, I was able to demonstrate that auditory cues evoked the same level of alertness across different populations and that differences in behavioral responses are not due to subjective differences of cue processing (Chapter 2). More importantly, I was able to show that verbal commands and auditory cues can be functionally discriminated by the brain. While both sounds are alerting they ought to be used complementary, depending on the intended goal (Chapter 3). The studies that employed the looming sound to redirect spatial attention to an unattended visual target showed a robust benefit in response times at longer cue-target intervals (Chapter 4 and 5). The looming benefit in processing visual targets is also apparent as enhanced neural activity in the right posterior hemisphere 280ms after target onset. Source-estimation results suggest that a preferential activation of frontal and parietal areas, which are involved in attention orienting, give rise to this looming benefit (Chapter 5). Finally, auditory cues improved performance for unattended targets but might also benefit the central visuo-motor task by only directing attention to the periphery without moving the eyes away from the visuo-motor task. This demonstrates that auditory cues also help in managing attention by preparing for task switches such that covert attention is allocated to the respective task when this task has to be performed. Overall this dissertation demonstrates that the careful selection of auditory cues can go a long way in supporting attention management

EEG Characterization of Sensorimotor Networks: Implications in Stroke

The purpose of this dissertation was to use electroencephalography (EEG) to characterize sensorimotor networks and examine the effects of stroke on sensorimotor networks. Sensorimotor networks play an essential role in completion of everyday tasks, and when damaged, as in stroke survivors, the successful completion of seemingly simple motor tasks becomes fantasy. When sensorimotor networks are impaired as a result of stroke, varying degrees of sensorimotor deficits emerge, most often including loss of sensation and difficulty generating upper extremity movements. Although sensory therapies, such as the application of tendon vibration, have been shown to reduce the sensorimotor deficits after stroke, the underlying sensorimotor mechanisms associated with such improvements are unknown. While sensorimotor networks have been studied extensively, unanswered questions still surround their role in basic control paradigms and how their role changes after stroke. EEG provides a way to probe the high-speed temporal dynamics of sensorimotor networks that other more common imaging modalities lack. Sensorimotor network function was examined in controls during a task designed to differentiate potential mechanisms of arm stabilization and determine to what degree the sensorimotor network is involved. After sensorimotor network function was characterized in controls, we examined the effect of stroke on the sensorimotor network during rest and described the reorganization that occurs. Lastly, we explored tendon vibration as a sensory therapy for stroke survivors and determined if sensorimotor network mechanisms underlie improvements in arm tracking performance due to wrist tendon vibration. We observed cortical activity and connectivity that suggests sensorimotor networks are involved in the control of arm stability, cortical networks reorganize to more asymmetric, local networks after stroke, and tendon vibration normalizes sensorimotor network activity and connectivity during motor control after stroke. This dissertation was among the first studies using EEG to characterize the high-speed temporal dynamics of sensorimotor networks following stroke. This new knowledge has led to a better understanding of how sensorimotor networks function under ordinary circumstances as well as extreme situations such as stroke and revealed previously unknown mechanisms by which tendon vibration improves motor control in stroke survivors, which will lead to better therapeutic approaches

Visual Attention-Related Processing: Perspectives from Ageing, Cognitive Decline and Dementia

Visual attention is essential for environmental interactions, but our ability to respond to stimuli gradually declines across the lifespan, and such deficits are even more pronounced in various states of cognitive impairment. Examining the integrity of related components, from elements of attention capture to executive control, will improve our understanding of related declines by helping to explain behavioural and neural effects, which will ultimately contribute towards our knowledge of the extent of dysfunctional attention processes and their impact upon everyday life. Accordingly, this Special Issue represents a body of literature that fundamentally advances insights into visual attention processing, featuring studies spanning healthy ageing, mild cognitive impairment, and dementi

Multimodal Features for Detection of Driver Stress and Fatigue: Review

Driver fatigue and stress significantly contribute to higher number of car accidents worldwide. Although, different detection approaches have been already commercialized and used by car producers (and third party companies), research activities in this field are still needed in order to increase the reliability of these alert systems. Also, in the context of automated driving, the driver mental state assessment will be an important part of cars in future. This paper presents state-of-the-art review of different approaches for driver fatigue and stress detection and evaluation. We describe in details various signals (biological, car and video) and derived features used for these tasks and we discuss their relevance and advantages. In order to make this review complete, we also describe different datasets, acquisition systems and experiment scenarios