Systems engineering approaches to safety in transport systems

openDuring driving, driver behavior monitoring may provide useful information to prevent road traffic accidents caused by driver distraction. It has been shown that 90% of road traffic accidents are due to human error and in 75% of these cases human error is the only cause. Car manufacturers have been interested in driver monitoring research for several years, aiming to enhance the general knowledge of driver behavior and to evaluate the functional state as it may drastically influence driving safety by distraction, fatigue, mental workload and attention. Fatigue and sleepiness at the wheel are well known risk factors for traffic accidents. The Human Factor (HF) plays a fundamental role in modern transport systems. Drivers and transport operators control a vehicle towards its destination in according to their own sense, physical condition, experience and ability, and safety strongly relies on the HF which has to take the right decisions. On the other hand, we are experiencing a gradual shift towards increasingly autonomous vehicles where HF still constitutes an important component, but may in fact become the "weakest link of the chain", requiring strong and effective training feedback. The studies that investigate the possibility to use biometrical or biophysical signals as data sources to evaluate the interaction between human brain activity and an electronic machine relate to the Human Machine Interface (HMI) framework. The HMI can acquire human signals to analyse the specific embedded structures and recognize the behavior of the subject during his/her interaction with the machine or with virtual interfaces as PCs or other communication systems. Based on my previous experience related to planning and monitoring of hazardous material transport, this work aims to create control models focused on driver behavior and changes of his/her physiological parameters. Three case studies have been considered using the interaction between an EEG system and external device, such as driving simulators or electronical components. A case study relates to the detection of the driver's behavior during a test driver. Another case study relates to the detection of driver's arm movements according to the data from the EEG during a driver test. The third case is the setting up of a Brain Computer Interface (BCI) model able to detect head movements in human participants by EEG signal and to control an electronic component according to the electrical brain activity due to head turning movements. Some videos showing the experimental results are available at https://www.youtube.com/channel/UCj55jjBwMTptBd2wcQMT2tg.openXXXIV CICLO - INFORMATICA E INGEGNERIA DEI SISTEMI/ COMPUTER SCIENCE AND SYSTEMS ENGINEERING - Ingegneria dei sistemiZero, Enric

Proceedings of the 3rd International Mobile Brain/Body Imaging Conference : Berlin, July 12th to July 14th 2018

The 3rd International Mobile Brain/Body Imaging (MoBI) conference in Berlin 2018 brought together researchers from various disciplines interested in understanding the human brain in its natural environment and during active behavior. MoBI is a new imaging modality, employing mobile brain imaging methods like the electroencephalogram (EEG) or near infrared spectroscopy (NIRS) synchronized to motion capture and other data streams to investigate brain activity while participants actively move in and interact with their environment. Mobile Brain / Body Imaging allows to investigate brain dynamics accompanying more natural cognitive and affective processes as it allows the human to interact with the environment without restriction regarding physical movement. Overcoming the movement restrictions of established imaging modalities like functional magnetic resonance tomography (MRI), MoBI can provide new insights into the human brain function in mobile participants. This imaging approach will lead to new insights into the brain functions underlying active behavior and the impact of behavior on brain dynamics and vice versa, it can be used for the development of more robust human-machine interfaces as well as state assessment in mobile humans.DFG, GR2627/10-1, 3rd International MoBI Conference 201

Neural Prosthetic Advancement: identification of circuitry in the Posterior Parietal Cortex

There are limited options for rehabilitation following an established Spinal Cord Injury (SCI) resulting in paralysis. For most of the individuals affected, SCI means a lifetime of confinement to a wheelchair and overall reduced independence. Brain-Computer and Brain-Machine Interface (BCI and BMI) techniques may be of aid when used for assistive purposes. However, these techniques are still far from being implemented in daily rehabilitative practice. Existing literature on the use of BCI and BMI techniques in SCI is limited and focuses on the extraction of motor control signals from the primary motor cortex (M1). However, evidence suggests that in long-term established SCI the functional activation of motor and premotor areas tends to decrease over time. In the present project, we explore the possibility of successful implementation of assistive BCI and BMI systems using posterior parietal areas as extraction sites of motor control activity. Firstly, we will investigate the representation of space in the posterior parietal cortex (PPC) and whether evidence of body-centered reference frames can be found in healthy individuals. We will then proceed to extract information regarding the residual level of motor imagery activity in individuals suffering from long-term and high-level SCI. Our aim is to ascertain whether functional activation of motor and posterior areas is comparable to that of matched controls. Finally, we will present work that was done in collaboration with the Netherlands Organisation for Applied Scientific Research that can offer an example of successful application of a BCI technique for rehabilitation purposes

Valutazione degli stati mentali attraverso l'utilizzo di interfacce cervello-computer passive

The monitoring of cognitive functions aims at gaining information about the current cognitive state of the user by decoding brain signals. In recent years, this approach allowed to acquire valuable information about the cognitive aspects regarding the interaction of humans with external world. From this consideration, researchers started to consider passive application of brain–computer interface (BCI) in order to provide a novel input modality for technical systems solely based on brain activity. The objective of this thesis is to demonstrate how the passive Brain Computer Interfaces (BCIs) applications can be used to assess the mental states of the users, in order to improve the human machine interaction. Two main studies has been proposed. The first one allows to investigate whatever the Event Related Potentials (ERPs) morphological variations can be used to predict the users’ mental states (e.g. attentional resources, mental workload) during different reactive BCI tasks (e.g. P300-based BCIs), and if these information can predict the subjects’ performance in performing the tasks. In the second study, a passive BCI system able to online estimate the mental workload of the user by relying on the combination of the EEG and the ECG biosignals has been proposed. The latter study has been performed by simulating an operative scenario, in which the occurrence of errors or lack of performance could have significant consequences. The results showed that the proposed system is able to estimate online the mental workload of the subjects discriminating three different difficulty level of the tasks ensuring a high reliability.La valutazione delle funzioni cognitive ha l’obbiettivo di ottenere informazioni sullo stato mentale attuale dell'utente, attraverso la decodifica dei segnali cerebrali. Negli ultimi anni, questo approccio ha consentito di indagare informazioni preziose sugli aspetti cognitivi riguardanti l'interazione tra l’uomo ed il mondo esterno. In base a queste considerazioni, recentemente si è considerata in letteratura la possibilità di utilizzare le interfacce cervello computer passive (BCI passivi) per interagire con dispositivi esterni, sfruttando l’attività spontanea dell’utente. L'obiettivo di questa tesi è quello di dimostrare come le interfacce cervello computer passive possano essere utilizzate per valutare lo stato mentale dell’utente, al fine di migliorare l'interazione uomo-macchina. Sono stati presentati due studi principali. Il primo ha l’obbiettivo di investigare le variazioni morfologiche dei potenziali evento correlati (ERP), al fine di associarle agli stati mentali dell’utente (es. attenzione, carico di lavoro mentale) durante l’utilizzo di BCI reattive, e come predittori delle performance raggiunte dai soggetti. Nel secondo studio è stato sviluppato e validato un sistema BCI passivo in grado di stimare il carico di lavoro mentale dell'utente durante task operative, attraverso la combinazione del segnale elettroencefalografico (EEG) ed elettrocardiografico (ECG). Quest'ultimo studio è stato effettuato simulando uno scenario operativo, in cui il verificarsi di errori da parte dell’operatore o il calo di prestazioni poteva avere conseguenze importanti. I risultati hanno mostrato che il sistema proposto è in grado di discriminare il carico di lavoro mentale percepito dall’utente su tre livelli di difficoltà, garantendo un’elevata affidabilità

Auditory cues for attention management

An exhaustible supply of mental resources necessitate that we are selective for what we attend to. Attention prioritizes what ought to be processed and what ignored, allocating valuable resources to selected information at the cost of unattended information elsewhere. For this purpose it is necessary to know the conditions that help the brain decide when attention should be paid, where to and to what information. The question that is central to this dissertation is how auditory cues can support the management of limited attentional resources based on auditory characteristics. Auditory cues can (1) increase the overall alertness, (2) orient attention to unattended information, or (3) manage attentional resources by informing of an upcoming task-switch and, therefore, indicate when to pay attention to which task. The first study of this dissertation investigated whether different population groups might process auditory cues differently, thus resulting in different levels of alertness (1). Study two examined more specifically whether the type of auditory cue (verbal command or auditory icon) used as in-vehicle notifications can influence the level of alertness (1). Studies three and four investigated the use of a special auditory cue characteristic, the looming intensity profile, for directing attention to regions of interest (2). Here, attention orienting to peripheral events was tested within a dual-task paradigm which required attention shifts between the two tasks (3). Throughout the studies, I show that electroencephalography (EEG) is an indispensable tool for evaluating auditory cues and their influence on crossmodal attention. By using EEG measurements, I was able to demonstrate that auditory cues evoked the same level of alertness across different populations and that differences in behavioral responses are not due to subjective differences of cue processing (Chapter 2). More importantly, I was able to show that verbal commands and auditory cues can be functionally discriminated by the brain. While both sounds are alerting they ought to be used complementary, depending on the intended goal (Chapter 3). The studies that employed the looming sound to redirect spatial attention to an unattended visual target showed a robust benefit in response times at longer cue-target intervals (Chapter 4 and 5). The looming benefit in processing visual targets is also apparent as enhanced neural activity in the right posterior hemisphere 280ms after target onset. Source-estimation results suggest that a preferential activation of frontal and parietal areas, which are involved in attention orienting, give rise to this looming benefit (Chapter 5). Finally, auditory cues improved performance for unattended targets but might also benefit the central visuo-motor task by only directing attention to the periphery without moving the eyes away from the visuo-motor task. This demonstrates that auditory cues also help in managing attention by preparing for task switches such that covert attention is allocated to the respective task when this task has to be performed. Overall this dissertation demonstrates that the careful selection of auditory cues can go a long way in supporting attention management

Ubiquitous Technologies for Emotion Recognition

Emotions play a very important role in how we think and behave. As such, the emotions we feel every day can compel us to act and influence the decisions and plans we make about our lives. Being able to measure, analyze, and better comprehend how or why our emotions may change is thus of much relevance to understand human behavior and its consequences. Despite the great efforts made in the past in the study of human emotions, it is only now, with the advent of wearable, mobile, and ubiquitous technologies, that we can aim to sense and recognize emotions, continuously and in real time. This book brings together the latest experiences, findings, and developments regarding ubiquitous sensing, modeling, and the recognition of human emotions

Predicting Sleepiness from Driving Behaviour

This research investigates the use of objective EEG analysis to determine multiple levels of sleepiness in drivers. In the literature, current methods propose a binary (awake or sleep) or ternary (awake, drowsy or sleep) classification of sleepiness. Having few classification of sleepiness increases the risk of the driver reaching dangerous levels of sleepiness before a safety system can prevent it. Also, these methods are based on subjective analysis of physiological variables, which leads to lack of reproducibility and loss of data, when a lack of consensus is reached amongst the EEG experts. Therefore, the doctoral challenge was to determine whether multiple levels of sleepiness could be defined with high accuracy, using an objective analysis of EEG, a reliable indicator of sleepiness. The study identified awake, post-awake, pre-sleep and sleep as the multiple levels of sleepiness through the objective analysis of EEG. The research used Neural Networks, a type of Machine Learning algorithm, to determine the accuracy of the proposed multiple levels of sleepiness. The Neural Networks were trained using driving and physiological behaviour. The EEG data and the driving and physiological variables were obtained through a series of experiments aimed to induce sleepiness, conducted in the driving simulator at the University of Leeds. As the Neural Network obtained high accuracy when differentiating between awake and sleep and between post-awake and pre-sleep, it led to the conclusion that the proposed objective classification based on objective EEG analysis was suitable. However, this study did not reach the highest levels of accuracy when the 4 levels of sleepiness are combined, nevertheless the solutions proposed by the researcher to be carried in future work can contribute towards increasing the accuracy of the proposed method

Applied and laboratory-based autonomic and neurophysiological monitoring during sustained attention tasks

Fluctuations during sustained attention can cause momentary lapses in performance which can have a significant impact on safety and wellbeing. However, it is less clear how unrelated tasks impact current task processes, and whether potential disturbances can be detected by autonomic and central nervous system measures in naturalistic settings. In a series of five experiments, I sought to investigate how prior attentional load impacts semi-naturalistic tasks of sustained attention, and whether neurophysiological and psychophysiological monitoring of continuous task processes and performance could capture attentional lapses. The first experiment explored various non-invasive electrophysiological and subjective methods during multitasking. The second experiment employed a manipulation of multitasking, task switching, to attempt to unravel the negative lasting impacts of multitasking on neural oscillatory activity, while the third experiment employed a similar paradigm in a semi-naturalistic environment of simulated driving. The fourth experiment explored the feasibility of measuring changes in autonomic processing during a naturalistic sustained monitoring task, autonomous driving, while the fifth experiment investigated the visual demands and acceptability of a biological based monitoring system. The results revealed several findings. While the first experiment demonstrated that only self-report ratings were able to successfully disentangle attentional load during multitasking; the second and third experiment revealed deficits in parieto-occipital alpha activity and continuous performance depending on the attentional load of a previous unrelated task. The fourth experiment demonstrated increased sympathetic activity and a smaller distribution of fixations during an unexpected event in autonomous driving, while the fifth experiment revealed the acceptability of a biological based monitoring system although further research is needed to unpick the effects on attention. Overall, the results of this thesis help to provide insight into how autonomic and central processes manifest during semi-naturalistic sustained attention tasks. It also provides support for a neuro- or biofeedback system to improve safety and wellbeing