Valutazione degli stati mentali attraverso l'utilizzo di interfacce cervello-computer passive

The monitoring of cognitive functions aims at gaining information about the current cognitive state of the user by decoding brain signals. In recent years, this approach allowed to acquire valuable information about the cognitive aspects regarding the interaction of humans with external world. From this consideration, researchers started to consider passive application of brain–computer interface (BCI) in order to provide a novel input modality for technical systems solely based on brain activity. The objective of this thesis is to demonstrate how the passive Brain Computer Interfaces (BCIs) applications can be used to assess the mental states of the users, in order to improve the human machine interaction. Two main studies has been proposed. The first one allows to investigate whatever the Event Related Potentials (ERPs) morphological variations can be used to predict the users’ mental states (e.g. attentional resources, mental workload) during different reactive BCI tasks (e.g. P300-based BCIs), and if these information can predict the subjects’ performance in performing the tasks. In the second study, a passive BCI system able to online estimate the mental workload of the user by relying on the combination of the EEG and the ECG biosignals has been proposed. The latter study has been performed by simulating an operative scenario, in which the occurrence of errors or lack of performance could have significant consequences. The results showed that the proposed system is able to estimate online the mental workload of the subjects discriminating three different difficulty level of the tasks ensuring a high reliability.La valutazione delle funzioni cognitive ha l’obbiettivo di ottenere informazioni sullo stato mentale attuale dell'utente, attraverso la decodifica dei segnali cerebrali. Negli ultimi anni, questo approccio ha consentito di indagare informazioni preziose sugli aspetti cognitivi riguardanti l'interazione tra l’uomo ed il mondo esterno. In base a queste considerazioni, recentemente si è considerata in letteratura la possibilità di utilizzare le interfacce cervello computer passive (BCI passivi) per interagire con dispositivi esterni, sfruttando l’attività spontanea dell’utente. L'obiettivo di questa tesi è quello di dimostrare come le interfacce cervello computer passive possano essere utilizzate per valutare lo stato mentale dell’utente, al fine di migliorare l'interazione uomo-macchina. Sono stati presentati due studi principali. Il primo ha l’obbiettivo di investigare le variazioni morfologiche dei potenziali evento correlati (ERP), al fine di associarle agli stati mentali dell’utente (es. attenzione, carico di lavoro mentale) durante l’utilizzo di BCI reattive, e come predittori delle performance raggiunte dai soggetti. Nel secondo studio è stato sviluppato e validato un sistema BCI passivo in grado di stimare il carico di lavoro mentale dell'utente durante task operative, attraverso la combinazione del segnale elettroencefalografico (EEG) ed elettrocardiografico (ECG). Quest'ultimo studio è stato effettuato simulando uno scenario operativo, in cui il verificarsi di errori da parte dell’operatore o il calo di prestazioni poteva avere conseguenze importanti. I risultati hanno mostrato che il sistema proposto è in grado di discriminare il carico di lavoro mentale percepito dall’utente su tre livelli di difficoltà, garantendo un’elevata affidabilità

Sound-production Related Cognitive Tasks for Onset Detection in Self-Paced Brain-Computer Interfaces

Objective. The main goal of this research is proposing a novel method of onset detection for Self-Paced (SP) Brain-Computer Interfaces (BCIs) to increase usability and practicality of BCIs towards real-world uses from laboratory research settings. Approach. To achieve this goal, various Sound-Production Related Cognitive Tasks (SPRCTs) were tested against idle state in offline and simulated-online experiments. An online experiment was then conducted that turned a messenger dialogue on when a new message arrived by executing the Sound Imagery (SI) onset detection task in real-life scenarios (e.g. watching video, reading text). The SI task was chosen as an onset task because of its advantages over other tasks: 1) Intuitiveness. 2) Beneficial for people with motor disabilities. 3) No significant overlap with other common, spontaneous cognitive states becoming easier to use in daily-life situations. 4) No dependence on user’s mother language. Main results. The final online experimental results showed the new SI onset task had significantly better performance than the Motor Imagery (MI) approach. 84.04% (SI) vs 66.79% (MI) TFP score for sliding image scenario, 80.84% vs 61.07% for watching video task. Furthermore, the onset response speed showed the SI task being significantly faster than MI. In terms of usability, 75% of subjects answered SI was easier to use. Significance. The new SPRCT outperforms typical MI for SP onset detection BCIs (significantly better performance, faster onset response and easier usability), therefore it would be more easily used in daily-life situations. Another contribution of this thesis is a novel EMG artefact-contaminated EEG channel selection and handling method that showed significant class separation improvement against typical blind source separation techniques. A new performance evaluation metric for SP BCIs, called true-false positive score was also proposed as a standardised performance assessment method that considers idle period length, which was not considered in other typical metrics

Brain-Computer Interfaces for Non-clinical (Home, Sports, Art, Entertainment, Education, Well-being) Applications

HCI researchers interest in BCI is increasing because the technology industry is expanding into application areas where efficiency is not the main goal of concern. Domestic or public space use of information and communication technology raise awareness of the importance of affect, comfort, family, community, or playfulness, rather than efficiency. Therefore, in addition to non-clinical BCI applications that require efficiency and precision, this Research Topic also addresses the use of BCI for various types of domestic, entertainment, educational, sports, and well-being applications. These applications can relate to an individual user as well as to multiple cooperating or competing users. We also see a renewed interest of artists to make use of such devices to design interactive art installations that know about the brain activity of an individual user or the collective brain activity of a group of users, for example, an audience. Hence, this Research Topic also addresses how BCI technology influences artistic creation and practice, and the use of BCI technology to manipulate and control sound, video, and virtual and augmented reality (VR/AR)

Decoding mental representations for neurofeedback in motor and imagery tasks

The inability to execute coordinated movements can have a profound impact on an individual’s ability to complete the activities of daily living. Often times, these impairments arise as a result of neurological injury such as a stroke. Conventional physical therapy of the affected limb is often ineffective, possibly because it treats the symptom rather than the cause of the problem: impaired motor control circuitry in the brain. A potential method to address this issue is to employ neuroimaging technology to guide neuroplastic changes in the brain to restore motor function. The first aim of this dissertation was to develop and validate the use of a neurofeedback training target derived from functionally aligned data of healthy individuals. Our results showed a significant improvement in classification accuracy of individual finger presses when group data was aligned based on function rather than anatomy. This indicates that our functionally aligned template could provide an effective target for neural reinforcement of finger individuation beyond traditional anatomically aligned templates. For more severely impaired individuals, neuroimaging can also be used to develop assistive technologies by creating a connection between the brain and an external device, a technology known as a Brain-Computer Interface (BCI). In this work, we sought to address some of the current gaps in BCI usability by investigating the implementation of a visual imagery control paradigm. The second aim in this dissertation attempted to characterize the flexibility of visual imagery by mapping out the neural representational space of imagery for multiple image categories. While we were able to significantly identify when an individual was engaged in visual imagery compared to resting, our classifier was unable to accurately discern between the category-level neural activity. Our third aim expounded on this topic by investigating some of the key experimental components that contribute to successful performance. The results showed that visual imagery following a perception cue provided a stronger, more easily classifiable signal than visual imagery following an auditory cue. These results indicate that visual imagery is a challenging, yet possible BCI paradigm that could be useful in situations where other methods are ill suited.Mechanical Engineerin

Applications of realtime fMRI for non-invasive brain computer interface-decoding and neurofeedback

Non-invasive brain-computer interfaces (BCIs) seek to enable or restore brain function by using neuroimaging e.g. functional magnetic resonance imaging (fMRI), to engage brain activations without the need for explicit behavioural output or surgical implants. Brain activations are converted into output signals, for use in communication interfaces, motor prosthetics, or to directly shape brain function via a feedback loop. The aim of this thesis was to develop cognitive BCIs using realtime fMRI (rt-fMRI), with the potential for use as a communication interface, or for initiating neural plasticity to facilitate neurorehabilitation. Rt-fMRI enables brain activation to be manipulated directly to produce changes in function, such as perception. Univariate and multivariate classification approaches were used to decode brain activations produced by the deployment of covert spatial attention to simple visual stimuli. Primary and higher order visual areas were examined, as well as potential control regions. The classification platform was then developed to include the use of real-world visual stimuli, exploiting the use of category-specific visual areas, and demonstrating real-world applicability as a communications interface. Online univariate classification of spatial attention was successfully achieved, with individual classification accuracies for 4-quadrant spatial attention reaching 70%. Further, a novel implementation of m-sequences enabled the use of the timing of stimuli presentation to enhance signal characterisation. An established rt-fMRI analysis loop was then used for neurofeedback-led manipulation of category-specific visual brain regions, modulating their functioning, and, as a result, biasing visual perception during binocular rivalry. These changes were linked with functional and effective connectivity changes in trained regions, as well as in a putative top-down control region. The work presented provides proof-of-principle for non-invasive BCIs using rt-fMRI, with the potential for translation into the clinical environment. Decoding and 4 neurofeedback applied to non-invasive and implantable BCIs form an evolving continuum of options for enabling and restoring brain function

On Tackling Fundamental Constraints in Brain-Computer Interface Decoding via Deep Neural Networks

A Brain-Computer Interface (BCI) is a system that provides a communication and control medium between human cortical signals and external devices, with the primary aim to assist or to be used by patients who suffer from a neuromuscular disease. Despite significant recent progress in the area of BCI, there are numerous shortcomings associated with decoding Electroencephalography-based BCI signals in real-world environments. These include, but are not limited to, the cumbersome nature of the equipment, complications in collecting large quantities of real-world data, the rigid experimentation protocol and the challenges of accurate signal decoding, especially in making a system work in real-time. Hence, the core purpose of this work is to investigate improving the applicability and usability of BCI systems, whilst preserving signal decoding accuracy. Recent advances in Deep Neural Networks (DNN) provide the possibility for signal processing to automatically learn the best representation of a signal, contributing to improved performance even with a noisy input signal. Subsequently, this thesis focuses on the use of novel DNN-based approaches for tackling some of the key underlying constraints within the area of BCI. For example, recent technological improvements in acquisition hardware have made it possible to eliminate the pre-existing rigid experimentation procedure, albeit resulting in noisier signal capture. However, through the use of a DNN-based model, it is possible to preserve the accuracy of the predictions from the decoded signals. Moreover, this research demonstrates that by leveraging DNN-based image and signal understanding, it is feasible to facilitate real-time BCI applications in a natural environment. Additionally, the capability of DNN to generate realistic synthetic data is shown to be a potential solution in reducing the requirement for costly data collection. Work is also performed in addressing the well-known issues regarding subject bias in BCI models by generating data with reduced subject-specific features. The overall contribution of this thesis is to address the key fundamental limitations of BCI systems. This includes the unyielding traditional experimentation procedure, the mandatory extended calibration stage and sustaining accurate signal decoding in real-time. These limitations lead to a fragile BCI system that is demanding to use and only suited for deployment in a controlled laboratory. Overall contributions of this research aim to improve the robustness of BCI systems and enable new applications for use in the real-world