20 research outputs found

    Valutazione degli stati mentali attraverso l'utilizzo di interfacce cervello-computer passive

    Get PDF
    The monitoring of cognitive functions aims at gaining information about the current cognitive state of the user by decoding brain signals. In recent years, this approach allowed to acquire valuable information about the cognitive aspects regarding the interaction of humans with external world. From this consideration, researchers started to consider passive application of brain–computer interface (BCI) in order to provide a novel input modality for technical systems solely based on brain activity. The objective of this thesis is to demonstrate how the passive Brain Computer Interfaces (BCIs) applications can be used to assess the mental states of the users, in order to improve the human machine interaction. Two main studies has been proposed. The first one allows to investigate whatever the Event Related Potentials (ERPs) morphological variations can be used to predict the users’ mental states (e.g. attentional resources, mental workload) during different reactive BCI tasks (e.g. P300-based BCIs), and if these information can predict the subjects’ performance in performing the tasks. In the second study, a passive BCI system able to online estimate the mental workload of the user by relying on the combination of the EEG and the ECG biosignals has been proposed. The latter study has been performed by simulating an operative scenario, in which the occurrence of errors or lack of performance could have significant consequences. The results showed that the proposed system is able to estimate online the mental workload of the subjects discriminating three different difficulty level of the tasks ensuring a high reliability.La valutazione delle funzioni cognitive ha l’obbiettivo di ottenere informazioni sullo stato mentale attuale dell'utente, attraverso la decodifica dei segnali cerebrali. Negli ultimi anni, questo approccio ha consentito di indagare informazioni preziose sugli aspetti cognitivi riguardanti l'interazione tra l’uomo ed il mondo esterno. In base a queste considerazioni, recentemente si è considerata in letteratura la possibilità di utilizzare le interfacce cervello computer passive (BCI passivi) per interagire con dispositivi esterni, sfruttando l’attività spontanea dell’utente. L'obiettivo di questa tesi è quello di dimostrare come le interfacce cervello computer passive possano essere utilizzate per valutare lo stato mentale dell’utente, al fine di migliorare l'interazione uomo-macchina. Sono stati presentati due studi principali. Il primo ha l’obbiettivo di investigare le variazioni morfologiche dei potenziali evento correlati (ERP), al fine di associarle agli stati mentali dell’utente (es. attenzione, carico di lavoro mentale) durante l’utilizzo di BCI reattive, e come predittori delle performance raggiunte dai soggetti. Nel secondo studio è stato sviluppato e validato un sistema BCI passivo in grado di stimare il carico di lavoro mentale dell'utente durante task operative, attraverso la combinazione del segnale elettroencefalografico (EEG) ed elettrocardiografico (ECG). Quest'ultimo studio è stato effettuato simulando uno scenario operativo, in cui il verificarsi di errori da parte dell’operatore o il calo di prestazioni poteva avere conseguenze importanti. I risultati hanno mostrato che il sistema proposto è in grado di discriminare il carico di lavoro mentale percepito dall’utente su tre livelli di difficoltà, garantendo un’elevata affidabilità

    Sound-production Related Cognitive Tasks for Onset Detection in Self-Paced Brain-Computer Interfaces

    Get PDF
    Objective. The main goal of this research is proposing a novel method of onset detection for Self-Paced (SP) Brain-Computer Interfaces (BCIs) to increase usability and practicality of BCIs towards real-world uses from laboratory research settings. Approach. To achieve this goal, various Sound-Production Related Cognitive Tasks (SPRCTs) were tested against idle state in offline and simulated-online experiments. An online experiment was then conducted that turned a messenger dialogue on when a new message arrived by executing the Sound Imagery (SI) onset detection task in real-life scenarios (e.g. watching video, reading text). The SI task was chosen as an onset task because of its advantages over other tasks: 1) Intuitiveness. 2) Beneficial for people with motor disabilities. 3) No significant overlap with other common, spontaneous cognitive states becoming easier to use in daily-life situations. 4) No dependence on user’s mother language. Main results. The final online experimental results showed the new SI onset task had significantly better performance than the Motor Imagery (MI) approach. 84.04% (SI) vs 66.79% (MI) TFP score for sliding image scenario, 80.84% vs 61.07% for watching video task. Furthermore, the onset response speed showed the SI task being significantly faster than MI. In terms of usability, 75% of subjects answered SI was easier to use. Significance. The new SPRCT outperforms typical MI for SP onset detection BCIs (significantly better performance, faster onset response and easier usability), therefore it would be more easily used in daily-life situations. Another contribution of this thesis is a novel EMG artefact-contaminated EEG channel selection and handling method that showed significant class separation improvement against typical blind source separation techniques. A new performance evaluation metric for SP BCIs, called true-false positive score was also proposed as a standardised performance assessment method that considers idle period length, which was not considered in other typical metrics

    Applications of realtime fMRI for non-invasive brain computer interface-decoding and neurofeedback

    Get PDF
    Non-invasive brain-computer interfaces (BCIs) seek to enable or restore brain function by using neuroimaging e.g. functional magnetic resonance imaging (fMRI), to engage brain activations without the need for explicit behavioural output or surgical implants. Brain activations are converted into output signals, for use in communication interfaces, motor prosthetics, or to directly shape brain function via a feedback loop. The aim of this thesis was to develop cognitive BCIs using realtime fMRI (rt-fMRI), with the potential for use as a communication interface, or for initiating neural plasticity to facilitate neurorehabilitation. Rt-fMRI enables brain activation to be manipulated directly to produce changes in function, such as perception. Univariate and multivariate classification approaches were used to decode brain activations produced by the deployment of covert spatial attention to simple visual stimuli. Primary and higher order visual areas were examined, as well as potential control regions. The classification platform was then developed to include the use of real-world visual stimuli, exploiting the use of category-specific visual areas, and demonstrating real-world applicability as a communications interface. Online univariate classification of spatial attention was successfully achieved, with individual classification accuracies for 4-quadrant spatial attention reaching 70%. Further, a novel implementation of m-sequences enabled the use of the timing of stimuli presentation to enhance signal characterisation. An established rt-fMRI analysis loop was then used for neurofeedback-led manipulation of category-specific visual brain regions, modulating their functioning, and, as a result, biasing visual perception during binocular rivalry. These changes were linked with functional and effective connectivity changes in trained regions, as well as in a putative top-down control region. The work presented provides proof-of-principle for non-invasive BCIs using rt-fMRI, with the potential for translation into the clinical environment. Decoding and 4 neurofeedback applied to non-invasive and implantable BCIs form an evolving continuum of options for enabling and restoring brain function

    On Tackling Fundamental Constraints in Brain-Computer Interface Decoding via Deep Neural Networks

    Get PDF
    A Brain-Computer Interface (BCI) is a system that provides a communication and control medium between human cortical signals and external devices, with the primary aim to assist or to be used by patients who suffer from a neuromuscular disease. Despite significant recent progress in the area of BCI, there are numerous shortcomings associated with decoding Electroencephalography-based BCI signals in real-world environments. These include, but are not limited to, the cumbersome nature of the equipment, complications in collecting large quantities of real-world data, the rigid experimentation protocol and the challenges of accurate signal decoding, especially in making a system work in real-time. Hence, the core purpose of this work is to investigate improving the applicability and usability of BCI systems, whilst preserving signal decoding accuracy. Recent advances in Deep Neural Networks (DNN) provide the possibility for signal processing to automatically learn the best representation of a signal, contributing to improved performance even with a noisy input signal. Subsequently, this thesis focuses on the use of novel DNN-based approaches for tackling some of the key underlying constraints within the area of BCI. For example, recent technological improvements in acquisition hardware have made it possible to eliminate the pre-existing rigid experimentation procedure, albeit resulting in noisier signal capture. However, through the use of a DNN-based model, it is possible to preserve the accuracy of the predictions from the decoded signals. Moreover, this research demonstrates that by leveraging DNN-based image and signal understanding, it is feasible to facilitate real-time BCI applications in a natural environment. Additionally, the capability of DNN to generate realistic synthetic data is shown to be a potential solution in reducing the requirement for costly data collection. Work is also performed in addressing the well-known issues regarding subject bias in BCI models by generating data with reduced subject-specific features. The overall contribution of this thesis is to address the key fundamental limitations of BCI systems. This includes the unyielding traditional experimentation procedure, the mandatory extended calibration stage and sustaining accurate signal decoding in real-time. These limitations lead to a fragile BCI system that is demanding to use and only suited for deployment in a controlled laboratory. Overall contributions of this research aim to improve the robustness of BCI systems and enable new applications for use in the real-world

    Short- and Long-Term Changes in Attention, Memory and Brain Activity Following Exercise, Motor Learning, and Expertise

    Get PDF
    How humans perceive, embody, and execute actions has been an area of intense study in cognitive neuroscience, and these investigations shed light on how we adaptively learn from and interact with an ever-changing world. All of the knowledge associated with action, including sensorimotor representations, the words we use to describe them, and the memories that store this information, are represented in distributed brain regions that comprise knowledge schemas. With repeated practice or training, one can acquire a highly specialized motor repertoire that fosters even more efficient and adaptable behaviour to achieve peak performance. Using behavioural, EEG and fMRI approaches, I will present a series of investigations that evaluate the impact of short-term exercise and long-term dance practice on the development of expert knowledge schemas. In Chapters 2 and 3, I will demonstrate that activating one domain in the schema (e.g., action processing) will prime other domains (e.g., verbal attention and working memory) to induce translational performance improvements. Subsequent chapters will reveal how familiarity with a specific genre of dance influences behavioural (Chapter 3) and neurophysiological signatures of action perception, how these motor representations are coded in sensorimotor association areas (Chapters 4), and how they change with repeated practice and performance (Chapter 5). How these findings contribute to our model of expert knowledge schemas will be discussed in Chapter 6. These findings bear efficacy for the therapeutic application of exercise and dance programs to alleviate motor, cognitive and neurophysiological impairments in several clinical populations, including people with Parkinsons disease

    Towards simultaneous electroencephalography and functional near-infrared spectroscopy for improving diagnostic accuracy in prolonged disorders of consciousness: a healthy cohort study

    Get PDF
    Qualitative clinical assessments of the recovery of awareness after severe brain injury require an assessor to differentiate purposeful behaviour from spontaneous behaviour. As many such behaviours are minimal and inconsistent, behavioural assessments are susceptible to diagnostic errors. Advanced neuroimaging tools such as functional magnetic resonance imaging and electroencephalography (EEG) can bypass behavioural responsiveness and reveal evidence of covert awareness and cognition within the brains of some patients, thus providing a means for more accurate diagnoses, more accurate prognoses, and, in some instances, facilitated communication. As each individual neuroimaging method has its own advantages and disadvantages (e.g., signal resolution, accessibility, etc.), this thesis studies on healthy individuals a burgeoning technique of non-invasive electrical and optical neuroimaging—simultaneous EEG and functional near-infrared spectroscopy (fNIRS)—that can be applied at the bedside. Measuring reliable covert behaviours is correlated with participant engagement, instrumental sensitivity and the accurate localisation of responses, aspects which are further addressed over three studies. Experiment 1 quantifies the typical EEG changes in response to covert commands in the absence and presence of an object. This is investigated to determine whether a goal-directed task can yield greater EEG control accuracy over simple monotonous imagined single-joint actions. Experiment 2 characterises frequency domain NIRS changes in response to overt and covert hand movements. A method for reconstructing haemodynamics using the less frequently investigated phase parameter is outlined and the impact of noise contaminated NIRS measurements are discussed. Furthermore, classification performances between frequency-domain and continuous-wave-like signals are compared. Experiment 3 lastly applies these techniques to determine the potential of simultaneous EEG-fNIRS classification. Here a sparse channel montage that would ultimately favour clinical utility is used to demonstrate whether such a hybrid method containing rich spatial and temporal information can improve the classification of covert responses in comparison to unimodal classification of signals. The findings and discussions presented within this thesis identify a direction for future research in order to more accurately translate the brain state of patients with a prolonged disorder of consciousness

    Affective Computing

    Get PDF
    This book provides an overview of state of the art research in Affective Computing. It presents new ideas, original results and practical experiences in this increasingly important research field. The book consists of 23 chapters categorized into four sections. Since one of the most important means of human communication is facial expression, the first section of this book (Chapters 1 to 7) presents a research on synthesis and recognition of facial expressions. Given that we not only use the face but also body movements to express ourselves, in the second section (Chapters 8 to 11) we present a research on perception and generation of emotional expressions by using full-body motions. The third section of the book (Chapters 12 to 16) presents computational models on emotion, as well as findings from neuroscience research. In the last section of the book (Chapters 17 to 22) we present applications related to affective computing
    corecore