Synthesizing Speech from Intracranial Depth Electrodes using an Encoder-Decoder Framework

Speech Neuroprostheses have the potential to enable communication for people with dysarthria or anarthria. Recent advances have demonstrated high-quality text decoding and speech synthesis from electrocorticographic grids placed on the cortical surface. Here, we investigate a less invasive measurement modality in three participants, namely stereotactic EEG (sEEG) that provides sparse sampling from multiple brain regions, including subcortical regions. To evaluate whether sEEG can also be used to synthesize high-quality audio from neural recordings, we employ a recurrent encoder-decoder model based on modern deep learning methods. We find that speech can indeed be reconstructed with correlations up to 0.8 from these minimally invasive recordings, despite limited amounts of training data

Decoding executed and imagined grasping movements from distributed non-motor brain areas using a Riemannian decoder

Using brain activity directly as input for assistive tool control can circumventmuscular dysfunction and increase functional independence for physically impaired people. The motor cortex is commonly targeted for recordings, while growing evidence shows that there exists decodable movement-related neural activity outside of the motor cortex. Several decoding studies demonstrated significant decoding from distributed areas separately. Here, we combine information from all recorded non-motor brain areas and decode executed and imagined movements using a Riemannian decoder. We recorded neural activity from 8 epilepsy patients implanted with stereotactic-electroencephalographic electrodes (sEEG), while they performed an executed and imagined grasping tasks. Before decoding, we excluded all contacts in or adjacent to the central sulcus. The decoder extracts a low-dimensional representation of varying number of components, and classified move/no-move using a minimum-distance-to-geometric-mean Riemannian classifier. We show that executed and imagined movements can be decoded from distributed non-motor brain areas using a Riemannian decoder, reaching an area under the receiver operator characteristic of 0.83 ± 0.11. Furthermore, we highlight the distributedness of the movement-related neural activity, as no single brain area is the main driver of performance. Our decoding results demonstrate a first application of a Riemannian decoder on sEEG data and show that it is able to decode from distributed brain-wide recordings outside of the motor cortex. This brief report highlights the perspective to explore motor-related neural activity beyond the motor cortex, as many areas contain decodable information

Correspondence between large-scale ictal and interictal epileptic networks revealed by single photon emission computed tomography (SPECT) and electroencephalography (EEG)-functional magnetic resonance imaging (fMRI)

OBJECTIVE: Epilepsy is increasingly recognized as a network disorder, but the spatial relationship between ictal and interictal networks is still largely unexplored. In this work, we compared hemodynamic changes related to seizures and interictal spikes on a whole brain scale. METHODS: Twenty-eight patients with refractory focal epilepsy (14 temporal and 14 extratemporal lobe) underwent both subtraction ictal single photon emission computed tomography (SPECT) coregistered to magnetic resonance imaging (MRI) (SISCOM) and spike-related electroencephalography (EEG-functional MRI (fMRI). SISCOM visualized relative perfusion changes during seizures, whereas EEG-fMRI mapped blood oxygen level-dependent (BOLD) changes related to spikes. Similarity between statistical maps of both modalities was analyzed per patient using the following two measures: (1) correlation between unthresholded statistical maps (Pearson's correlation coefficient) and (2) overlap between thresholded images (Dice coefficient). Overlap was evaluated at a regional level, for hyperperfusions and activations and for hypoperfusions and deactivations separately, using different thresholds. Nonparametric permutation tests were applied to assess statistical significance (p ≤ 0.05). RESULTS: We found significant and positive correlations between hemodynamic changes related to seizures and spikes in 27 (96%) of 28 cases (median correlation coefficient 0.29 [range -0.12 to 0.62]). In 20 (71%) of 28 cases, spatial overlap between hyperperfusion on SISCOM and activation on EEG-fMRI was significantly larger than expected by chance. Congruent changes were not restricted to the territory of the presumed epileptogenic zone, but could be seen at distant sites (e.g., cerebellum and basal ganglia). Overlap between ictal hypoperfusion and interictal deactivation was statistically significant in 22 (79%) of 28 patients. Despite the high rate of congruence, discrepancies were observed for both modalities. SIGNIFICANCE: We conclude that hemodynamic changes related to seizures and spikes varied spatially with the same sign and within a common network. Overlap was present in regions nearby and distant from discharge origin.status: publishe

Sensitivity and Specificity of Interictal EEG-fMRI for Detecting the Ictal Onset Zone at Different Statistical Thresholds

There is currently a lack of knowledge about electroencephalography (EEG)-functional magnetic resonance imaging (fMRI) specificity. Our aim was to define sensitivity and specificity of blood oxygen level dependent (BOLD) responses to interictal epileptic spikes during EEG-fMRI for detecting the ictal onset zone (IOZ). We studied 21 refractory focal epilepsy patients who had a well-defined IOZ after a full presurgical evaluation and interictal spikes during EEG-fMRI. Areas of spike-related BOLD changes overlapping the IOZ in patients were considered as true positives; if no overlap was found, they were treated as false-negatives. Matched healthy case-controls had undergone similar EEG-fMRI in order to determine true-negative and false-positive fractions. The spike-related regressor of the patient was used in the design matrix of the healthy case-control. Suprathreshold BOLD changes in the brain of controls were considered as false positives, absence of these changes as true negatives. Sensitivity and specificity were calculated for different statistical thresholds at the voxel level combined with different cluster size thresholds and represented in receiver operating characteristic (ROC)-curves. Additionally, we calculated the ROC-curves based on the cluster containing the maximal significant activation. We achieved a combination of 100% specificity and 62% sensitivity, using a Z-threshold in the interval 3.4-3.5 and cluster size threshold of 350 voxels. We could obtain higher sensitivity at the expense of specificity. Similar performance was found when using the cluster containing the maximal significant activation. Our data provide a guideline for different EEG-fMRI settings with their respective sensitivity and specificity for detecting the IOZ. The unique cluster containing the maximal significant BOLD activation was a sensitive and specific marker of the IOZ.status: publishe

A reliable and time-saving semiautomatic spike-template-based analysis of interictal EEG-fMRI

A prerequisite for the implementation of interictal electroencephalography-correlated functional magnetic resonance imaging (EEG-fMRI) in the presurgical work-up for epilepsy surgery is straightforward processing. We propose a new semi-automatic method as alternative for the challenging and time-consuming visual spike identification.status: publishe

Epilepsy and Neurodevelopmental Comorbidities in Tuberous Sclerosis Complex: A Natural History Study

BACKGROUND: We studied the natural history, genotype influence, and inter-relationship of epilepsy and neuropsychiatric disorders in tuberous sclerosis complex. METHODS: Patients were identified using the TSC Natural History Database, the largest repository of longitudinally studied patients enrolled by the TSC Clinics Consortium. RESULTS: A cohort of 1657 TSC Natural History Database patients was analyzed. Eighty-eight percent patients (91% TSC2 vs 82% TSC1; P = 0.002) had epilepsy; TSC2 was more frequent with epilepsy onset at age less than two years (TSC2 82% vs TSC1 54%; P < 0.001) and infantile spasms (TSC2 56% vs TSC1 27%; P < 0.001). Frequency of intellectual disability (intelligence quotient less than 70) was higher when epilepsy coexisted (P < 0.001), but was not impacted by genotype (P = 0.08). Severe intellectual disability (intelligence quotient less than 50) was associated with epilepsy onset at age less than two years (P = 0.007), but not with the epilepsy duration (P = 0.45). Autism was diagnosed in 23% and was associated with epilepsy (P < 0.001), particularly with epilepsy onset at age less than two years (P = 0.02) but not with genotype (P = 0.06). Attention-deficit/hyperactivity disorder (age greater than four years) was diagnosed in 18% and was associated with epilepsy (P < 0.001), but genotype made no difference. Nine percent had anxiety (age greater than seven years) and 6% had depression (age greater than nine years), but neither showed association with epilepsy or genotype. CONCLUSIONS: Epilepsy is associated with intellectual disability, and when epilepsy begins before age two years the frequency and severity of intellectual disability is much higher. Epilepsy is also associated with autism and attention-deficit/hyperactivity disorder but not with anxiety and depression. Additional trials, blinded, prospective, and adequately powered, will help clarify if early and effective treatment of epilepsy may also mitigate intellectual disability, autism, and attention-deficit/hyperactivity disorder.status: publishe

Augmenting interictal mapping with neurovascular coupling biomarkers by structured factorization of epileptic EEG and fMRI data

EEG-correlated fMRI analysis is widely used to detect regional BOLD fluctuations that are synchronized to interictal epileptic discharges, which can provide evidence for localizing the ictal onset zone. However, the typical, asymmetrical and mass-univariate approach cannot capture the inherent, higher order structure in the EEG data, nor multivariate relations in the fMRI data, and it is nontrivial to accurately handle varying neurovascular coupling over patients and brain regions. We aim to overcome these drawbacks in a data-driven manner by means of a novel structured matrix-tensor factorization: the single-subject EEG data (represented as a third-order spectrogram tensor) and fMRI data (represented as a spatiotemporal BOLD signal matrix) are jointly decomposed into a superposition of several sources, characterized by space-time-frequency profiles. In the shared temporal mode, Toeplitz-structured factors account for a spatially specific, neurovascular ‘bridge’ between the EEG and fMRI temporal fluctuations, capturing the hemodynamic response's variability over brain regions. By analyzing interictal data from twelve patients, we show that the extracted source signatures provide a sensitive localization of the ictal onset zone (10/12). Moreover, complementary parts of the IOZ can be uncovered by inspecting those regions with the most deviant neurovascular coupling, as quantified by two entropy-like metrics of the hemodynamic response function waveforms (9/12). Hence, this multivariate, multimodal factorization provides two useful sets of EEG-fMRI biomarkers, which can assist the presurgical evaluation of epilepsy. We make all code required to perform the computations available at https://github.com/svaneynd/structured-cmtf.</p