Towards empathic neurofeedback for interactive storytelling

Interactive Narrative is a form of digital entertainment based on AI techniques which support narrative generation and user interaction. Despite recent progress in the field, there is still a lack of unified models integrating narrative generation, user response and interaction. This paper addresses this issue by revisiting existing Interactive Narrative paradigms, granting explicit status to users’ disposition towards story characters. We introduce a novel Brain-Computer Interface (BCI) design, which attempts to capture empathy for the main character in a way that is compatible with filmic theories of emotion. Results from two experimental studies with a fully-implemented system demonstrate the effectiveness of a neurofeedback-based approach, showing that subjects can successfully modulate their emotional support for a character who is confronted with challenging situations. A preliminary fMRI analysis also shows activation during user interaction, in regions of the brain associated with emotional control

Dual array EEG-fMRI : An approach for motion artifact suppression in EEG recorded simultaneously with fMRI

Objective: Although simultaneous recording of EEG and MRI has gained increasing popularity in recent years, the extent of its clinical use remains limited by various technical challenges. Motion interference is one of the major challenges in EEG-fMRI. Here we present an approach which reduces its impact with the aid of an MR compatible dual-array EEG (daEEG) in which the EEG itself is used both as a brain signal recorder and a motion sensor. Methods: We implemented two arrays of EEG electrodes organized into two sets of nearly orthogonally intersecting wire bundles. The EEG was recorded using referential amplifiers inside a 3 T MR-scanner. Virtual bipolar measurements were taken both along bundles (creating a small wire loop and therefore minimizing artifact) and across bundles (creating a large wire loop and therefore maximizing artifact). Independent component analysis (ICA) was applied. The resulting ICA components were classified into brain signal and noise using three criteria: 1) degree of two-dimensional spatial correlation between ICA coefficients along bundles and across bundles; 2) amplitude along bundles vs. across bundles; 3) correlation with ECG. The components which passed the criteria set were transformed back to the channel space. Motion artifact suppression and the ability to detect interictal epileptic spikes following daEEG and Optimal Basis Set (OBS) procedures were compared in 10 patients with epilepsy. Results: The SNR achieved by daEEG was 11.05 +/- 3.10 and by OBS was 8.25 +/- 1.01 (p <0.00001). In 9 of 10 patients, more spikes were detected after daEEG than after OBS (p <0.05). Significance: daEEG improves signal quality in EEG-fMRI recordings, expanding its clinical and research potential. (C) 2016 Elsevier Inc. All rights reserved.Peer reviewe

Neural and functional validation of fMRI-informed EEG model of right inferior frontal gyrus activity

The right inferior frontal gyrus (rIFG) is a region involved in the neural underpinning of cognitive control across several domains such as inhibitory control and attentional allocation process. Therefore, it constitutes a desirable neural target for brain-guided interventions such as neurofeedback (NF). To date, rIFG-NF has shown beneficial ability to rehabilitate or enhance cognitive functions using functional Magnetic Resonance Imaging (fMRI-NF). However, the utilization of fMRI-NF for clinical purposes is severely limited, due to its poor scalability. The present study aimed to overcome the limited applicability of fMRI-NF by developing and validating an EEG model of fMRI-defined rIFG activity (hereby termed ''Electrical FingerPrint of rIFG''; rIFG-EFP). To validate the computational model, we employed two experiments in healthy individuals. The first study (n = 14) aimed to test the target engagement of the model by employing rIFG-EFP-NF training while simultaneously acquiring fMRI. The second study (n = 41) aimed to test the functional outcome of two sessions of rIFG-EFP-NF using a risk preference task (known to depict cognitive control processes), employed before and after the training. Results from the first study demonstrated neural target engagement as expected, showing associated rIFG-BOLD signal changing during simultaneous rIFG-EFP-NF training. Target anatomical specificity was verified by showing a more precise prediction of the rIFG-BOLD by the rIFG-EFP model compared to other EFP models. Results of the second study suggested that successful learning to up-regulate the rIFG-EFP signal through NF can reduce one's tendency for risk taking, indicating improved cognitive control after two sessions of rIFG-EFP-NF. Overall, our results confirm the validity of a scalable NF method for targeting rIFG activity by using an EEG probe

One-Class FMRI-Inspired EEG Model for Self-Regulation Training.

Recent evidence suggests that learned self-regulation of localized brain activity in deep limbic areas such as the amygdala, may alleviate symptoms of affective disturbances. Thus far self-regulation of amygdala activity could be obtained only via fMRI guided neurofeedback, an expensive and immobile procedure. EEG on the other hand is relatively inexpensive and can be easily implemented in any location. However the clinical utility of EEG neurofeedback for affective disturbances remains limited due to low spatial resolution, which hampers the targeting of deep limbic areas such as the amygdala. We introduce an EEG prediction model of amygdala activity from a single electrode. The gold standard used for training is the fMRI-BOLD signal in the amygdala during simultaneous EEG/fMRI recording. The suggested model is based on a time/frequency representation of the EEG data with varying time-delay. Previous work has shown a strong inhomogeneity among subjects as is reflected by the models created to predict the amygdala BOLD response from EEG data. In that work, different models were constructed for different subjects. In this work, we carefully analyzed the inhomogeneity among subjects and were able to construct a single model for the majority of the subjects. We introduce a method for inhomogeneity assessment. This enables us to demonstrate a choice of subjects for which a single model could be derived. We further demonstrate the ability to modulate brain-activity in a neurofeedback setting using feedback generated by the model. We tested the effect of the neurofeedback training by showing that new subjects can learn to down-regulate the signal amplitude compared to a sham group, which received a feedback obtained by a different participant. This EEG based model can overcome substantial limitations of fMRI-NF. It can enable investigation of NF training using multiple sessions and large samples in various locations

Comparison between the cEFP and the EFP performances.

<p>a) Depicts the performance of the individual model constructed for the first session when testing on the second session. It compared with the cEFP performance on the same sessions. The results are an average over subjects whose first session was included in the common model construction process (<i>n</i> = 9). b) Compares the cEFP performance with the performance of two ‘optimal’ EFPs, when applied to a group of new subjects (<i>n</i> = 18, 9 subjects). In Fig 5a and 5b, the star's color represents the method that obtained significance (*<i>p</i> < 0.05). The error bars are standard deviations over sessions. c) Depicts the cEFP percentage change histogram (relative to EFP).</p

The transformation steps before applying the metric.

<p>a) The original EFP. b) Expanding y-axes to a minimum resolution of 1Hz. c) Collapsing y-axes to a uniform frequency band division. d) Reshaping EFP to a vector.</p

Down-regulating the common EFP signal amplitude.

<p>a) Mean results of the amygdala common EFP-NF. The <i>y</i> axis shows the mean cEFP amplitude during BL (left columns) and NF (right columns). Only the test group (red columns, <i>n</i> = 7) had significantly reduced cEFP amplitude during NF relative to BL (F(1,11) = 24.46, **<i>p</i><0.01). b) Individual results of the common EFP-NF. The <i>y</i> axis shows the cEFP amplitude during NF and the <i>x</i> axis shows the cEFP amplitude during BL. Markers (red = test; blue = sham) below the diagonal represent subjects that during NF reduced cEFP activity relative to BL. 6 out of 7 subjects from the test group could significantly reduce cEFP activity during NF relative to BL compared with only 1 out 6 subjects in the sham group. <i>*p</i><0.05, <i>**p</i><0.01, and <i>n</i> = 13. For illustration purposes, the cEFP amplitude of the BL for each subject was multiplied by the NF mean. The actual range of the cEFP amplitude during BL was (-0.2)-(0.34).</p

Scheme of the common model construction framework.

<p>The samples were divided in a leave-one-out manner into training and testing sets. The training set was used for model selection and the testing set was used for model validation. An inner cross-validation was used for choosing the optimal model (i.e. finding the model coefficients and the best regularization parameter) based on regularized ridge-regression training. The training input was the time-frequency representation of the EEG data and the training target was the fMRI BOLD signal in the amygdala. Each time-point in the BOLD signal corresponded to a time-window in the EEG. The resultant model coefficients suggest frequency bands and time delays that correlate to the BOLD activity in the amygdala.</p