Application of neural activity based error detection for improvement of a continuous BMI control.

<p>Subjects intend to move a cursor towards the top left target (white arrow). If the decoding is correct, the cursor performs the intended movement and no neuronal error signal is elicited in a subject. If there is a discrepancy between the intended and the decoded movement, an ERNR can be elicited. If the discrepancy is large enough, it can elicit an execution ERNR. If the execution error is detected by the BMI system, the decoding algorithm can be adapted to reduce the number of errors in decoding in the future. If the unwanted movement causes the cursor to reach an unwanted target, an outcome ERNR may be evoked. If the outcome ERNR is detected by the BMI system, it can change the decoding algorithm as well, this time in a different way.</p

Classifier selection and performance evaluation of the error detection algorithm.

<p>(A) The dataset was divided into three parts of equal length: training, validation and testing. The training data set was used to build the detection algorithm. To optimize the parameters of the detection algorithm (see Methods), the detection performance was evaluated on the validation data set for different values of the parameters. The parameter values yielding the highest performance were used to build the detection classifier which was then applied on the testing data to evaluate its performance. (B) Classifier building: The classifier was built using signal features from error (green) and baseline epochs (white). Signal features were taken from LFC and HFC (<a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0055235#pone-0055235-g004" target="_blank">Figure 4</a>) at multiple time points and from multiple electrodes. (C) Performance evaluation: The classifier used to calculate probability of an error event in a sliding window fashion, across the continuous signals, in order to detect error events. Finally, performance was calculated by comparing the times of detected error events to the times of the real error events.</p

Detection results when using signals with or without MRNR subtraction.

<p><i>C_YX</i> for detection of outcome (top) or execution (bottom) errors when using signals with (red) and without (blue) MRNR subtraction from all electrodes. Different columns show <i>C_YX</i> when low frequency component (left), high frequency component (center) or both signal components (right) of the recorded signals were used for detection. Error bars show 95% confidence intervals. Detection was made using rLDA.</p

Number of recorded sessions and events for each of the subjects.

<p>Number of recorded sessions and events for each of the subjects.</p

Locations of ECoG grid electrodes in relation to the cortex.

<p>Electrode positions (red, green and black circles) were reconstructed from the post-implantation MRI scan and positioned over the pre-implantation MRI scan <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0055235#pone.0055235-Kovalev1" target="_blank">[88]</a>. For S1, S3 and S4 red (green) circles represent electrodes that showed motor (somatosensory) responses to electrical stimulation mapping (ESM). For S2, motor and somatosensory electrodes were determined from sulci reconstruction. Central sulci, Sylvian fissures and, for S2 only, pre and post central sulcus are shown as blue lines. These were drawn by hand to resemble sulci reconstruction from the post-implantation MRI scan. Each of the subjects was implanted with an 8×8 ECoG grid. In S2, no recordings were made from the top row of the ECoG grid. In addition to ECoG grids, the Figure shows two 6 electrode ECoG strips over the frontal lobe (FLa and FLb) for S1 and two 4 electrode ECoG strips (FBa and FBb) over the frontal lobe for S3. In this study, we analyzed the recording from the ECoG grids only.</p

Total dataset length for different temporal tolerances.

<p>Total dataset length for different temporal tolerances.</p

Task and error events.

<p>A: Screenshot of the paradigm as seen by the subjects on the computer screen. Subjects played a video game in which they moved a spaceship in the horizontal direction (left-right) to evade the blocks dropping from above. Every time the spaceship collided with a block (collision event; B) one life was lost. Collision events elicited outcome ERNR. From time to time, the spaceship moved in the opposite direction of the joystick movement for 500 ms (movement mismatch event; C). These events elicited execution errors.</p

Detection results when using signals from electrode sets of different sizes.

<p><i>C_YX</i> for detection of outcome (left) or execution (right) errors for different sizes of the electrode sets, maximized over all possible electrode subsets and averaged over subjects. We compared single electrodes (blue lines), electrode quartets (red lines) and the set of all grid electrodes (black lines) when using low frequency component (bottom), high frequency component (middle) or both frequency components (top) of the recorded signals. Detection was made using rLDA. Error bars show 95% confidence intervals.</p

Extraction of low and high frequency components (LFC and HFC) from the ECoG recordings.

<p>Recordings in every channel were first re-referenced using the common average over all recording channels that do not show epileptic activity. To get the LFC, the re-referenced signal was low-pass filtered using a Savitzky-Golay filter (symmetric, 2^nd order, 250 ms window length). To get the HFC, the re-referenced signal is transformed to the time-frequency space using short-time Fourier transform. The amplitudes of the transformed signal were then divided by the average baseline amplitude for every frequency bin separately. The HFC was defined as the average normalized amplitudes across all bins within the HFC frequency range (see Methods for a definition of the frequency ranges).To correct for movement related neuronal activity, MRNRs were subtracted.</p