Optimizing the face paradigm of BCI system by modified mismatch negative paradigm

Many recent studies have focused on improving the performance of event-related potential (ERP) based brain computer interfaces (BCIs). The use of a face pattern has been shown to obtain high classification accuracies and information transfer rates (ITRs) by evoking discriminative ERPs (N200 and N400) in addition to P300 potentials. Recently, it has been proved that the performance of traditional P300-based BCIs could be improved through a modification of the mismatch pattern. In this paper, a mismatch inverted face pattern (MIF-pattern) was presented to improve the performance of the inverted face pattern (IF-pattern), one of the state of the art patterns used in visual-based BCI systems. Ten subjects attended in this experiment. The result showed that the mismatch inverted face pattern could evoke significantly larger vertex positive potentials (p < 0.05) and N400s (p < 0.05) compared to the inverted face pattern. The classification accuracy (mean accuracy is 99.58%) and ITRs (mean bit rate is 27.88 bit/min) of the mismatch inverted face pattern was significantly higher than that of the inverted face pattern (p < 0.05)

Measuring Cognitive Conflict in Virtual Reality with Feedback-Related Negativity

As virtual reality (VR) emerges as a mainstream platform, designers have started to experiment new interaction techniques to enhance the user experience. This is a challenging task because designers not only strive to provide designs with good performance but also carefully ensure not to disrupt users' immersive experience. There is a dire need for a new evaluation tool that extends beyond traditional quantitative measurements to assist designers in the design process. We propose an EEG-based experiment framework that evaluates interaction techniques in VR by measuring intentionally elicited cognitive conflict. Through the analysis of the feedback-related negativity (FRN) as well as other quantitative measurements, this framework allows designers to evaluate the effect of the variables of interest. We studied the framework by applying it to the fundamental task of 3D object selection using direct 3D input, i.e. tracked hand in VR. The cognitive conflict is intentionally elicited by manipulating the selection radius of the target object. Our first behavior experiment validated the framework in line with the findings of conflict-induced behavior adjustments like those reported in other classical psychology experiment paradigms. Our second EEG-based experiment examines the effect of the appearance of virtual hands. We found that the amplitude of FRN correlates with the level of realism of the virtual hands, which concurs with the Uncanny Valley theory

Comparison of the ERP-Based BCI Performance Among Chromatic (RGB) Semitransparent Face Patterns

Objective: Previous studies have shown that combing with color properties may be used as part of the display presented to BCI users in order to improve performance. Build on this, we explored the effects of combinations of face stimuli with three primary colors (RGB) on BCI performance which is assessed by classification accuracy and information transfer rate (ITR). Furthermore, we analyzed the waveforms of three patterns. Methods: We compared three patterns in which semitransparent face is overlaid three primary colors as stimuli: red semitransparent face (RSF), green semitransparent face (GSF), and blue semitransparent face (BSF). Bayesian linear discriminant analysis (BLDA) was used to construct the individual classifier model. In addition, a Repeated-measures ANOVA (RM-ANOVA) and Bonferroni correction were chosen for statistical analysis. Results: The results indicated that the RSF pattern achieved the highest online averaged accuracy with 93.89%, followed by the GSF pattern with 87.78%, while the lowest performance was caused by the BSF pattern with an accuracy of 81.39%. Furthermore, significant differences in classification accuracy and ITR were found between RSF and GSF (p < 0.05) and between RSF and BSF patterns (p < 0.05). Conclusion: The semitransparent faces colored red (RSF) pattern yielded the best performance of the three patterns. The proposed patterns based on ERP-BCI system have a clinically significant impact by increasing communication speed and accuracy of the P300-speller for patients with severe motor impairment

Comparison of the BCI Performance between the Semitransparent Face Pattern and the Traditional Face Pattern

Brain-computer interface (BCI) systems allow users to communicate with the external world by recognizing the brain activity without the assistance of the peripheral motor nervous system. P300-based BCI is one of the most common used BCI systems that can obtain high classification accuracy and information transfer rate (ITR). Face stimuli can result in large event-related potentials and improve the performance of P300-based BCI. However, previous studies on face stimuli focused mainly on the effect of various face types (i.e., face expression, face familiarity, and multifaces) on the BCI performance. Studies on the influence of face transparency differences are scarce. Therefore, we investigated the effect of semitransparent face pattern (STF-P) (the subject could see the target character when the stimuli were flashed) and traditional face pattern (F-P) (the subject could not see the target character when the stimuli were flashed) on the BCI performance from the transparency perspective. Results showed that STF-P obtained significantly higher classification accuracy and ITR than those of F-P (p < 0.05)

COGNITIVE PROCESSING AND BRAIN COMMUNICATION IN AMYOTROPHIC LATERAL SCLEROSIS

Amyotrophic Lateral Sclerosis (ALS) is a fatal neurodegenerative disease characterized by progressive paralysis of limbs and bulbar musculature. This severe physical impairment makes cognitive evaluation a big challenge, thus there is a great need for an assessment that does not require overt motor responses. Moreover, we need of augmentative communication strategies because the disease generally leads to complete paralysis and, therefore, patients are unable to communicate with the external world by any means. For this purpose, Brain Computer Interfaces (BCIs) seem a promising approach to facilitate communication with these patients. The aim of this thesis is twofold. First, assessing cognitive processing in ALS by means of a novel evaluation tool. Second, allowing brain communication in completely paralyzed ALS patients who had lost their vision in order to eliminate the unbearable loss of communication in paralysis (“unlocking the locked-in”). The first study introduces a novel approach for assessing cognitive functions in ALS. This approach uses neuropsychological tests that require minimal overt motor or verbal responses; together with vibro-tactile P300s. Results indicate mild cognitive impairment in oral language comprehension tasks and reduced vibro-tactile P300 amplitudes in patients compared to healthy controls. Importantly, correlations between the vibro-tactile P300 latency and psychometric test results suggest that the former measure could serve as a neurophysiological marker of cognitive decline in ALS patients. The second study introduces a distraction paradigm based in auditory event-related potentials (ERPs) to evaluate the ability of change detection, focusing, and re-orientation of attention in ALS. The results revealed a modification of the amplitude and the latency of the N200, the P300 and the re-orienting negativity (RON) components. This could suggest an alteration of the endogenous mechanism that controls the detection of change, thus resulting in a reduction of the allocation and the re-orientation of attentional resources. The third study aimed at testing the feasibility of a Near Infrared Spectroscopy (NIRS) -based BCI communication approach for patients in the Completely Locked-in Stage (CLIS) due to ALS. For this purpose two CLIS patients were trained to control their cerebral-cortex´s functional-activations in response to auditory processing of correct or incorrect statements assessed with NIRS. The results of the study are very promising, showing that both CLIS patients communicated with fronto-cortical oxygenation based BCI at an average correct response rate of 70% over a period of several weeks. We conclude that this novel approach of brain-communication is safe and, reliable, representing, so far, the best communication possible for patients in completely locked-in state. In conclusion we propose a) the novel combination of vibro-tactile or acoustic ERPs and motor-independent neuropsychological tests as an alternative and easily implementable way for assessing cognitive functions in ALS and b) we confirm the usefulness and effectiveness of above mentioned electrophysiological approaches in the late stage of ALS either to assess cognitive processing or to establish communication with a BCI system

Event-related potentials to task-irrelevant changes in facial expressions

Abstract Background Numerous previous experiments have used oddball paradigm to study change detection. This paradigm is applied here to study change detection of facial expressions in a context which demands abstraction of the emotional expression-related facial features among other changing facial features. Methods Event-related potentials (ERPs) were recorded in adult humans engaged in a demanding auditory task. In an oddball paradigm, repeated pictures of faces with a neutral expression ('standard', p = .9) were rarely replaced by pictures with a fearful ('fearful deviant', p = .05) or happy ('happy deviant', p = .05) expression. Importantly, facial identities changed from picture to picture. Thus, change detection required abstraction of facial expression from changes in several low-level visual features. Results ERPs to both types of deviants differed from those to standards. At occipital electrode sites, ERPs to deviants were more negative than ERPs to standards at 150&#8211;180 ms and 280&#8211;320 ms post-stimulus. A positive shift to deviants at fronto-central electrode sites in the analysis window of 130&#8211;170 ms post-stimulus was also found. Waveform analysis computed as point-wise comparisons between the amplitudes elicited by standards and deviants revealed that the occipital negativity emerged earlier to happy deviants than to fearful deviants (after 140 ms versus 160 ms post-stimulus, respectively). In turn, the anterior positivity was earlier to fearful deviants than to happy deviants (110 ms versus 120 ms post-stimulus, respectively). Conclusion ERP amplitude differences between emotional and neutral expressions indicated pre-attentive change detection of facial expressions among neutral faces. The posterior negative difference at 150&#8211;180 ms latency resembled visual mismatch negativity (vMMN) &#8211; an index of pre-attentive change detection previously studied only to changes in low-level features in vision. The positive anterior difference in ERPs at 130&#8211;170 ms post-stimulus probably indexed pre-attentive attention orienting towards emotionally significant changes. The results show that the human brain can abstract emotion related features of faces while engaged to a demanding task in another sensory modality.peerReviewe