Broad-Band Visually Evoked Potentials: Re(con)volution in Brain-Computer Interfacing.

Brain-Computer Interfaces (BCIs) allow users to control devices and communicate by using brain activity only. BCIs based on broad-band visual stimulation can outperform BCIs using other stimulation paradigms. Visual stimulation with pseudo-random bit-sequences evokes specific Broad-Band Visually Evoked Potentials (BBVEPs) that can be reliably used in BCI for high-speed communication in speller applications. In this study, we report a novel paradigm for a BBVEP-based BCI that utilizes a generative framework to predict responses to broad-band stimulation sequences. In this study we designed a BBVEP-based BCI using modulated Gold codes to mark cells in a visual speller BCI. We defined a linear generative model that decomposes full responses into overlapping single-flash responses. These single-flash responses are used to predict responses to novel stimulation sequences, which in turn serve as templates for classification. The linear generative model explains on average 50% and up to 66% of the variance of responses to both seen and unseen sequences. In an online experiment, 12 participants tested a 6 × 6 matrix speller BCI. On average, an online accuracy of 86% was reached with trial lengths of 3.21 seconds. This corresponds to an Information Transfer Rate of 48 bits per minute (approximately 9 symbols per minute). This study indicates the potential to model and predict responses to broad-band stimulation. These predicted responses are proven to be well-suited as templates for a BBVEP-based BCI, thereby enabling communication and control by brain activity only

Layout optimization.

<p>To prevent cross-talk between neighbouring cells the allocation of bit-sequences to cells is optimized. An initial (left) and optimized (right) layout are shown. Numbers indicate bit-sequences. The shade indicates the correlation between responses to codes from neighbouring cells. The correlations are depicted between horizontal, vertical, and diagonal neighbours. For diagonal neighbours, the maximum correlation of the two diagonals is shown. In this perspective darker colours represent less correlation and better neighbours, hence an increased potential to distinguish.</p

Grand average responses.

<p>Grand averages of both spatially filtered ERPs (solid lines) and predicted responses (dashed lines) are shown. The quality of fit by generating the response to the same bit-sequence as reconvolution was trained on is shown at the top (<i>r</i>² = 0.343). The quality of fit by predicting the response to a bit-sequence that was not used during training is shown at the bottom (<i>r</i>² = 0.476).</p

Colour feedback.

<p>While a trial progresses, colour feedback is given regarding the classifier’s certainty. All cells start gray (A). A cell is coloured more green if the cell is more likely to be selected, whereas a cell is coloured more red when it is likely to not be selected. The colours are scaled to the specific margin and the maximum and minimum correlation between the single-trial and templates. Here, the target was ‘T’.</p

The BCI cycle.

<p>A general framework of BCIs, showing the consecutive steps in signal processing. These steps involve stimulation in a specific modality, data measurement and pre-processing, data analysis, and output or feedback generated by the BCI. Adapted from [<a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0133797#pone.0133797.ref001" target="_blank">1</a>].</p

The online pipeline.

<p>Three stages exist: training, calibration and testing. During training, responses <i>X</i> to stimuli from <i>V</i> are recorded. During calibration, <i>X</i> is deconvolved to pulse responses <i>r</i> using <i>V</i>. Template responses <i>T</i>_<i>V</i> and <i>T</i>_<i>U</i> are generated by convolving these <i>r</i> with the bit-sequences <i>V</i> and <i>U</i>, respectively. Templates are multiplied (circles) with filters (<i>W</i>_<i>X</i>, <i>W</i>_<i>T</i>) designed by CCA. The subset and layout are optimized giving <i>U</i>′ and </p><p></p><p></p><p></p><p><mi>T</mi><mi>U</mi><mo>′</mo></p><p></p><p></p><p></p>, and stopping margins <i>m</i> are learned. In the testing phase, a new single-trial <i>x</i> is assigned the class-label <i>y</i> that maximizes the correlation between the spatially filtered single-trial <i>x</i> and templates <p></p><p></p><p></p><p><mi>T</mi><mi>U</mi><mo>′</mo></p><p></p><p></p><p></p>. The classifier emits the class-label if the maximum correlation exceeds the threshold margin. In the case wherein the margin is not reached, more data is collected.<p></p

Platinum subsets.

<p>The steps for finding a subset of <i>p</i> = 3 codes from a set of <i>n</i> = 10 codes is shown. First the full set is clustered grouping similar points (A₁ till A₃). Then, iteratively (B till D) each cluster is collapsed into a single point by selecting one candidate. This candidate is chosen by maximizing the distance to all other living points outside the cluster (B₂, C₂, D₂). The remaining points form the Platinum subset (D₃).</p

Pulse responses.

<p>The spatially filtered pulse responses derived by the estimation step in reconvolution (left) and corresponding zero-padded power spectra (right) are shown for each participant. The top figures show the pulse responses on a short flash, the bottom ones show those for a long flash. The black bars represent the length of a single flash.</p

Stimulation sequences.

<p>Four modulated Gold codes are shown of <i>l</i> = 2*(2^<i>m</i> − 1) = 126 bits. These are pseudo-random binary-codes that are sequences of short on-off runs (i.e., ‘10’ or ‘100’) and long on-off runs (i.e., ‘110’ or ‘1100’). These on-off runs are used to modulate the luminance of the cells, and thus represent short and long flashes.</p