Rapid P300 brain-computer interface communication with a head-mounted display

Visual ERP (P300) based brain-computer interfaces (BCIs) allow for fast and reliable spelling and are intended as a muscle-independent communication channel for people with severe paralysis. However, they require the presentation of visual stimuli in the field of view of the user. A head-mounted display could allow convenient presentation of visual stimuli in situations, where mounting a conventional monitor might be difficult or not feasible (e.g., at a patient's bedside). To explore if similar accuracies can be achieved with a virtual reality (VR) headset compared to a conventional flat screen monitor, we conducted an experiment with 18 healthy participants. We also evaluated it with a person in the locked-in state (LIS) to verify that usage of the headset is possible for a severely paralyzed person. Healthy participants performed online spelling with three different display methods. In one condition a 5 x 5 letter matrix was presented on a conventional 22 inch TFT monitor. Two configurations of the VR headset were tested. In the first (glasses A), the same 5 x 5 matrix filled the field of view of the user. In the second (glasses B), single letters of the matrix filled the field of view of the user. The participant in the LIS tested the VR headset on three different occasions (glasses A condition only). For healthy participants, average online spelling accuracies were 94% (15.5 bits/min) using three flash sequences for spelling with the monitor and glasses A and 96% (16.2 bits/min) with glasses B. In one session, the participant in the LIS reached an online spelling accuracy of 100% (10 bits/min) using the glasses A condition. We also demonstrated that spelling with one flash sequence is possible with the VR headset for healthy users (mean: 32.1 bits/min, maximum reached by one user: 71.89 bits/min at 100% accuracy). We conclude that the VR headset allows for rapid P300 BCI communication in healthy users and may be a suitable display option for severely paralyzed persons

Evaluation of visual parameters to control a visual ERP-BCI under single-trial classification

A brain-computer interface (BCIs) based on event-related potentials (ERPs) is a technology that provides a communication channel between a device and a user through their brain activity. These systems could be used to assist and facilitate decision making in applications such as an air traffic controller (ATC). Thus, this work attempts to be an approximation to determine whether it is possible to detect the stimulus through a single presentation of a stimulus (single-trial classification) and furthermore, to evaluate the ef-fects of the type of stimulus to be detected, or not knowing the position of the stimulus appearance in an ERP-BCI. This experiment has involved six participants in four experimental conditions. Two conditions varied only in the type of stimulus used, faces (a type of stimulus that has shown high per-formance in previous ERP-BCI proposals) versus radar planes; and two con-ditions varied in the prior knowledge of where the stimulus would appear on the screen (knowing vs. not knowing). The results suggest that the use of single-trial classification could be adequate to correctly detect the desired stimulus using and ERP-BCI. In addition, the results reveal no significant effect on either of the two factors. Therefore, it seems that radar planes may be as suitable stimuli as faces and that not knowing the location of the target stimulus is not a significant problem, at least in a standard BCI scenario without distracting stimuli. Therefore, future studies should consider these findings for the design of an ATC using an ERP-BCI for stimulus detection.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Evaluation of flashing stimuli shape and colour heterogeneity using a P300 brain-computer interface speller

Objective: Previous works using a visual P300-based speller have reported an improvement modifying the shape or colour of the presented stimulus. However, the effects of both blended factors have not been yet studied. Thus, the aim of the present work was to study both factors and assess the interaction between them. Method: Fifteen na\"ive participants tested four different spellers in a calibration and online task. All spellers were similar except the employed illumination of the target stimulus: white letters, white blocks, coloured letters, and coloured blocks. Results: The block-shaped conditions offered an improvement versus the letter-shaped conditions in the calibration (accuracy) and online (accuracy and correct commands per minute) tasks. Analysis of the P300 waveform showed a larger difference between target and no target stimulus waveforms for the block-shaped conditions versus the letter-shaped. The hypothesis regarding the colour heterogeneity of the stimuli was not found at any level of the analysis. Conclusion: The use of block-shaped illumination demonstrated a better performance than the standard letter-shaped flashing stimuli in classification performance, correct commands per minute, and P300 waveform

Evaluation of Single-Trial Classification to Control a Visual ERP-BCI under a Situation Awareness Scenario.

An event-related potential (ERP)-based brain–computer interface (BCI) can be used to monitor a user’s cognitive state during a surveillance task in a situational awareness context. The present study explores the use of an ERP-BCI for detecting new planes in an air traffic controller (ATC). Two experiments were conducted to evaluate the impact of different visual factors on target detection. Experiment 1 validated the type of stimulus used and the effect of not knowing its appearance location in an ERP-BCI scenario. Experiment 2 evaluated the effect of the size of the target stimulus appearance area and the stimulus salience in an ATC scenario. The main results demonstrate that the size of the plane appearance area had a negative impact on the detection performance and on the amplitude of the P300 component. Future studies should address this issue to improve the performance of an ATC in stimulus detection using an ERP-BCI.Partial funding for open access charge: Universidad de Málag

Influence of spatial frequency in visual stimuli for cVEP-based BCIs: evaluation of performance and user experience

Code-modulated visual evoked potentials (c-VEPs) are an innovative control signal utilized in brain-computer interfaces (BCIs) with promising performance. Prior studies on steady-state visual evoked potentials (SSVEPs) have indicated that the spatial frequency of checkerboard-like stimuli influences both performance and user experience. Spatial frequency refers to the dimensions of the individual squares comprising the visual stimulus, quantified in cycles (i.e., number of black-white squares pairs) per degree of visual angle. However, the specific effects of this parameter on c-VEP-based BCIs remain unexplored. Therefore, the objective of this study is to investigate the role of spatial frequency of checkerboard-like visual stimuli in a c-VEP-based BCI. Sixteen participants evaluated selection matrices with eight spatial frequencies: C001 (0 c/°, 1×1 squares), C002 (0.15 c/°, 2×2 squares), C004 (0.3 c/°, 4×4 squares), C008 (0.6 c/°, 8×8 squares), C016 (1.2 c/°, 16×16 squares), C032 (2.4 c/°, 32×32 squares), C064 (4.79 c/°, 64×64 squares), and C128 (9.58 c/°, 128×128 squares). These conditions were tested in an online spelling task, which consisted of 18 trials each conducted on a 3×3 command interface. In addition to accuracy and information transfer rate (ITR), subjective measures regarding comfort, ocular irritation, and satisfaction were collected. Significant differences in performance and comfort were observed based on different stimulus spatial frequencies. Although all conditions achieved mean accuracy over 95% after 2.1 s of trial duration, C016 stood out in terms user experience. The proposed condition not only achieved a mean accuracy of 96.53% and 164.54 bits/min with a trial duration of 1.05s, but also was reported to be significantly more comfortable than the traditional C001 stimulus. Since both features are key for BCI development, higher spatial frequencies than the classical black-to-white stimulus might be more adequate for c-VEP systems. Hence, we assert that the spatial frequency should be carefully considered in the development of future applications for c-VEP-based BCIs

Do the stimuli of an SSVEP-based BCI really have to be the same as the stimuli used for training it?

International audienceDoes the stimulation used during the training on an SSVEP-based BCI have to be similar to that of the end use? We recorded six-channel EEG data from 12 subjects in various conditions of distance between targets, and of difference in color between targets. Our analysis revealed that the stimulation configuration used for training which leads to the best classification accuracy is not always the one which is closest to the end use configuration. We found that the distance between targets during training is of little influence if the end use targets are close to each other, but that training at far distance can lead to a better accuracy for far distance end use (p < .01). Additionally, an interaction effect is observed between training and testing color (p < .001): while training with monochrome targets leads to good performance only when the test context involves monochrome targets as well, a classifier trained on colored targets can be efficient for both colored and monochrome targets. In a nutshell, in the context of SSVEP-based BCI, training using distant targets of different colors seems to lead to the best and more robust performance in all end use contexts

Dual-Frequency SSVEP-based BCI for Reducing Eye Fatigue and Improving Classification Rate

학위논문 (박사)-- 서울대학교 대학원 : 공과대학 협동과정 바이오엔지니어링전공, 2016. 2. 박광석.The steady-state visual-evoked potential (SSVEP)-based brain-computer interface (BCI) has been widely investigated because of its high signal-to-noise ratio (SNR), and little requirement for training. However, the stimulus for evoking SSVEP causes high visual fatigue and has a risk of epileptic seizure. Furthermore, stimulation frequency is limited and the SSVEP amplitude diminishes when a monitor is used as a stimulator. In this thesis, a dual-frequency SSVEP is examined to resolve the aforementioned issues. Employing dual-frequency SSVEPs, two novel SSVEP-based BCIs are introduced to decrease eye fatigue and use harmonic frequencies with increased performance. First, the spectral characteristics of dual-frequency SSVEPs are investigated and frequency recognition methods for dual-frequency SSVEPs are suggested. Three methods based on power spectral density analysis (PSDA) and two methods based on canonical correlation analysis (CCA) were tested. The proposed CCA with a novel reference signal exhibited the best BCI performance, and the use of harmonic components improved the classification rate of the dual-frequency SSVEP. Second, the dual-frequency SSVEP response to an amplitude-modulated stimulus (AM-SSVEP) was explored to verify its performance with reduced eye fatigue. An amplitude-modulated stimulus was generated using the product of two sine waves at a carrier frequency (fc) and a modulating frequency (fm). The carrier frequency was higher than 40 Hz to reduce eye fatigue, and the modulating frequency ranged around the α-band (9–12 Hz) to utilize low-frequency harmonic information. The feasibility of AM-SSVEP with high BCI performance and low eye fatigue was confirmed through offline and online experiments. Using an optimized combination of the harmonic frequencies, the online experiments demonstrated that the accuracy of the AM-SSVEP was 97%, equivalent to that of the low-frequency SSVEP. Furthermore, subject evaluation indicated that an AM stimulus caused lower eye fatigue and less perception of flickering than a low-frequency stimulus, in a manner similar to a high-frequency stimulus. Third, a novel dual-frequency SSVEP-based hybrid SSVEP-P300 speller is introduced to overcome the frequency limitations and improve the performance. The hybrid speller consists of nine panels flickering at different frequencies. Each panel contains four different characters that appear in a random sequence. The flickering panel and the periodically updating character evoke the dual-frequency SSVEP, and the oddball stimulus of the target character evokes the P300. Ten subjects participated in offline and online experiments, in which accuracy and information transfer rate (ITR) were compared with those of conventional SSVEP and P300 spellers. The offline analysis revealed that the proposed speller elicited dual-frequency SSVEP. Moreover, the dual-frequency SSVEP significantly improved the SSVEP classification rate and ITR with a monitor in online experiments by 4 % accuracy and 18.8 bpm ITR. In conclusion, the proposed dual-frequency SSVEP-based BCIs reduce eye fatigue and improve SSVEP classification rate. The results indicate that this study provides a promising approach to make SSVEP-based BCIs more reliable and efficient for practical use.1. Introduction 1 1.1. Brain-Computer Interface 1 1.1.1. Basic Concepts 1 1.1.2. SSVEP-based BCIs 2 1.1.3. P300-based BCIs 5 1.1.4. Hybrid SSVEP-P300 BCIs 6 1.2. Motivation and Objectives 7 2. Frequency Recognition Methods for DFSSVEP-based BCI 11 2.1. Basic Concepts 11 2.2. DFSSVEP Recognition Methods 16 2.2.1. PSDA-based Methods 17 2.2.2. CCA-based Methods 20 2.3. Offline Analysis 23 2.3.1. Dual-Frequency Stimulus 23 2.3.2. Experimental Settings 24 2.3.3. Spectral Analysis of DFSSVEP 25 2.3.4. Signal Processing 26 2.4. Results 27 2.4.1. Harmonic Frequency 27 2.4.2. Comparison of Recognition Rates 28 2.5. Conclusion 31 3. DFSSVEP-based BCI for Reducing Eye Fatigue 33 3.1. Basic Concepts 33 3.1.1. Amplitude Modulation Technique 33 3.1.2. Amplitude-Modulated Stimuli for Evoking AM-SSVEP 35 3.2. Methods 38 3.2.1. Subjects and Experimental Settings 38 3.2.2. Offline Experiments 41 3.2.3. EEG Analysis 43 3.2.4. Online Experiments 45 3.3. Results 50 3.3.1. Harmonics of AM-SSVEP 50 3.3.2. Offline Analysis 54 3.3.3. CFC for Online Analysis 57 3.3.4. Online Analysis 59 3.3.5. Subject Evaluation 64 3.4. Discussion 66 3.4.1. Combining of Low- and High-Frequency SSVEPs 66 3.4.2. AM Harmonic Frequencies in CFC 70 3.4.3. Error Analysis 71 3.4.4. Effects of Environmental Illumination 74 3.5. Conclusion 76 4. DFSSVEP-based Hybrid BCI for Improving Classification Rate 79 4.1. Basic Concepts 79 4.2. Methods 85 4.2.1. Experimental Setting 85 4.2.2. Experimental Procedure 88 4.2.3. Signal Processing 89 4.2.4. Statistical Comparison of the EEG Responses 91 4.2.5. BCI Performance 92 4.3. Results 94 4.3.1. EEG Response to the Hybrid Speller 94 4.3.2. Offline Analysis 99 4.3.3. Online Analysis 102 4.4. Discussion 104 4.4.1. DFSSVEP 104 4.4.2. ITR Comparison with Conventional Spellers 109 4.4.3. ITR Comparison with Previous Studies 110 4.4.4. ITR with Different Visual Angle 114 4.4.5. Limitations 117 4.5. Conclusion 118 5. Conclusion 119 6. References 123 국문 초록 133Docto

The effect of leads on cognitive load and learning in a conceptually rich hypertext environment

The purpose of this experiment was to determine whether leads affect cognitive load and learning from conceptually rich hypertext. Measures of cognitive load included self-report of mental effort, reading time, and event-related desynchronization percentage of alpha, beta, and theta brain wave rhythms. Conceptual and structural knowledge tests, as well as a recall measure were used to determine learning performance. Measures of learners\u27 reading ability, prior knowledge, and metacognitive awareness were employed to establish the effect of individual differences on cognitive load and learning from traditional and lead-augmented hypertext. Results demonstrated that while leads appeared to reduce brain wave activity associated with split attention, processing of redundant information contained in hypertext nodes may have increased extraneous cognitive load, and decreased germane load that is required for learning to take place. Whereas the benefits of leads relative to cognitive load and learning may have been mediated by the redundancy effect, learners with better developed metacognitive skills tended to use leads as a tool to review information in the linked nodes while revisiting content in the primary text passage. Limitations of the currently available cognitive load measures are discussed as applied to direct assessment of this theoretical construct