Depth electrode neurofeedback with a virtual reality interface

Invasive brain–computer interfaces (BCI) provide better signal quality in terms of spatial localization, frequencies and signal/noise ratio, in addition to giving access to deep brain regions that play important roles in cognitive or affective processes. Despite some anecdotal attempts, little work has explored the possibility of integrating such BCI input into more sophisticated interactive systems like those which can be developed with game engines. In this article, we integrated an amygdala depth electrode recorder with a virtual environment controlling a virtual crowd. Subjects were asked to down regulate their amygdala using the level of unrest in the virtual room as feedback on how successful they were. We report early results which suggest that users adapt very easily to this paradigm and that the timing and fluctuations of amygdala activity during self-regulation can be matched by crowd animation in the virtual room. This suggests that depth electrodes could also serve as high-performance affective interfaces, notwithstanding their strictly limited availability, justified on medical grounds only

Towards empathic neurofeedback for interactive storytelling

Interactive Narrative is a form of digital entertainment based on AI techniques which support narrative generation and user interaction. Despite recent progress in the field, there is still a lack of unified models integrating narrative generation, user response and interaction. This paper addresses this issue by revisiting existing Interactive Narrative paradigms, granting explicit status to users’ disposition towards story characters. We introduce a novel Brain-Computer Interface (BCI) design, which attempts to capture empathy for the main character in a way that is compatible with filmic theories of emotion. Results from two experimental studies with a fully-implemented system demonstrate the effectiveness of a neurofeedback-based approach, showing that subjects can successfully modulate their emotional support for a character who is confronted with challenging situations. A preliminary fMRI analysis also shows activation during user interaction, in regions of the brain associated with emotional control

One-Class FMRI-Inspired EEG Model for Self-Regulation Training.

Recent evidence suggests that learned self-regulation of localized brain activity in deep limbic areas such as the amygdala, may alleviate symptoms of affective disturbances. Thus far self-regulation of amygdala activity could be obtained only via fMRI guided neurofeedback, an expensive and immobile procedure. EEG on the other hand is relatively inexpensive and can be easily implemented in any location. However the clinical utility of EEG neurofeedback for affective disturbances remains limited due to low spatial resolution, which hampers the targeting of deep limbic areas such as the amygdala. We introduce an EEG prediction model of amygdala activity from a single electrode. The gold standard used for training is the fMRI-BOLD signal in the amygdala during simultaneous EEG/fMRI recording. The suggested model is based on a time/frequency representation of the EEG data with varying time-delay. Previous work has shown a strong inhomogeneity among subjects as is reflected by the models created to predict the amygdala BOLD response from EEG data. In that work, different models were constructed for different subjects. In this work, we carefully analyzed the inhomogeneity among subjects and were able to construct a single model for the majority of the subjects. We introduce a method for inhomogeneity assessment. This enables us to demonstrate a choice of subjects for which a single model could be derived. We further demonstrate the ability to modulate brain-activity in a neurofeedback setting using feedback generated by the model. We tested the effect of the neurofeedback training by showing that new subjects can learn to down-regulate the signal amplitude compared to a sham group, which received a feedback obtained by a different participant. This EEG based model can overcome substantial limitations of fMRI-NF. It can enable investigation of NF training using multiple sessions and large samples in various locations

Comparison between the cEFP and the EFP performances.

<p>a) Depicts the performance of the individual model constructed for the first session when testing on the second session. It compared with the cEFP performance on the same sessions. The results are an average over subjects whose first session was included in the common model construction process (<i>n</i> = 9). b) Compares the cEFP performance with the performance of two ‘optimal’ EFPs, when applied to a group of new subjects (<i>n</i> = 18, 9 subjects). In Fig 5a and 5b, the star's color represents the method that obtained significance (*<i>p</i> < 0.05). The error bars are standard deviations over sessions. c) Depicts the cEFP percentage change histogram (relative to EFP).</p

The transformation steps before applying the metric.

<p>a) The original EFP. b) Expanding y-axes to a minimum resolution of 1Hz. c) Collapsing y-axes to a uniform frequency band division. d) Reshaping EFP to a vector.</p

Down-regulating the common EFP signal amplitude.

<p>a) Mean results of the amygdala common EFP-NF. The <i>y</i> axis shows the mean cEFP amplitude during BL (left columns) and NF (right columns). Only the test group (red columns, <i>n</i> = 7) had significantly reduced cEFP amplitude during NF relative to BL (F(1,11) = 24.46, **<i>p</i><0.01). b) Individual results of the common EFP-NF. The <i>y</i> axis shows the cEFP amplitude during NF and the <i>x</i> axis shows the cEFP amplitude during BL. Markers (red = test; blue = sham) below the diagonal represent subjects that during NF reduced cEFP activity relative to BL. 6 out of 7 subjects from the test group could significantly reduce cEFP activity during NF relative to BL compared with only 1 out 6 subjects in the sham group. <i>*p</i><0.05, <i>**p</i><0.01, and <i>n</i> = 13. For illustration purposes, the cEFP amplitude of the BL for each subject was multiplied by the NF mean. The actual range of the cEFP amplitude during BL was (-0.2)-(0.34).</p

Scheme of the common model construction framework.

<p>The samples were divided in a leave-one-out manner into training and testing sets. The training set was used for model selection and the testing set was used for model validation. An inner cross-validation was used for choosing the optimal model (i.e. finding the model coefficients and the best regularization parameter) based on regularized ridge-regression training. The training input was the time-frequency representation of the EEG data and the training target was the fMRI BOLD signal in the amygdala. Each time-point in the BOLD signal corresponded to a time-window in the EEG. The resultant model coefficients suggest frequency bands and time delays that correlate to the BOLD activity in the amygdala.</p

Scheme of the virtual machine used during EEG-NF training.

<p>The virtual machine receives the last 3 seconds in the EEG data and returns the predicted BOLD value that corresponds to the last change. a) New EEG segment is attached. b) Preprocessing of the buffer. c-d) Last 12 seconds are multiplied by the cEFP matrix coefficients to calculate the next point on the cEFP signal.</p