Unsupervised decoding of long-term, naturalistic human neural recordings with automated video and audio annotations

Fully automated decoding of human activities and intentions from direct neural recordings is a tantalizing challenge in brain-computer interfacing. Most ongoing efforts have focused on training decoders on specific, stereotyped tasks in laboratory settings. Implementing brain-computer interfaces (BCIs) in natural settings requires adaptive strategies and scalable algorithms that require minimal supervision. Here we propose an unsupervised approach to decoding neural states from human brain recordings acquired in a naturalistic context. We demonstrate our approach on continuous long-term electrocorticographic (ECoG) data recorded over many days from the brain surface of subjects in a hospital room, with simultaneous audio and video recordings. We first discovered clusters in high-dimensional ECoG recordings and then annotated coherent clusters using speech and movement labels extracted automatically from audio and video recordings. To our knowledge, this represents the first time techniques from computer vision and speech processing have been used for natural ECoG decoding. Our results show that our unsupervised approach can discover distinct behaviors from ECoG data, including moving, speaking and resting. We verify the accuracy of our approach by comparing to manual annotations. Projecting the discovered cluster centers back onto the brain, this technique opens the door to automated functional brain mapping in natural settings

ECoG Beta Suppression and Modulation During Finger Extension and Flexion

Neural oscillations originate predominantly from interacting cortical neurons and consequently reflect aspects of cortical information processing. However, their functional role is not yet fully understood and their interpretation is debatable. Amplitude modulations (AMs) in alpha (8–12 Hz), beta (13–30 Hz), and high gamma (70–150 Hz) band in invasive electrocorticogram (ECoG) and non-invasive electroencephalogram (EEG) signals change with behavior. Alpha and beta band AMs are typically suppressed (desynchronized) during motor behavior, while high gamma AMs highly correlate with the behavior. These two phenomena are successfully used for functional brain mapping and brain-computer interface (BCI) applications. Recent research found movement-phase related AMs (MPA) also in high beta/low gamma (24–40 Hz) EEG rhythms. These MPAs were found by separating the suppressed AMs into sustained and dynamic components. Sustained AM components are those with frequencies that are lower than the motor behavior. Dynamic components those with frequencies higher than the behavior. In this paper, we study ECoG beta/low gamma band (12–30 Hz/30–42 Hz) AM during repetitive finger movements addressing the question whether or not MPAs can be found in ECoG beta band. Indeed, MPA in the 12–18 Hz and 18–24 Hz band were found. This additional information may lead to further improvements in ECoG-based prediction and reconstruction of motor behavior by combining high gamma AM and beta band MPA

Interactive Web Application for Exploring Matrices of Neural Connectivity

We present here a browser-based application for visualizing patterns of connectivity in 3D stacked data matrices with large numbers of pairwise relations. Visualizing a connectivity matrix, looking for trends and patterns, and dynamically manipulating these values is a challenge for scientists from diverse fields, including neuroscience and genomics. In particular, high-dimensional neural data include those acquired via electroencephalography (EEG), electrocorticography (ECoG), magnetoencephalography (MEG), and functional MRI. Neural connectivity data contains multivariate attributes for each edge between different brain regions, which motivated our lightweight, open source, easy-to-use visualization tool for the exploration of these connectivity matrices to highlight connections of interest. Here we present a client-side, mobile-compatible visualization tool written entirely in HTML5/JavaScript that allows in-browser manipulation of user-defined files for exploration of brain connectivity. Visualizations can highlight different aspects of the data simultaneously across different dimensions. Input files are in JSON format, and custom Python scripts have been written to parse MATLAB or Python data files into JSON-loadable format. We demonstrate the analysis of connectivity data acquired via human ECoG recordings as a domain-specific implementation of our application. We envision applications for this interactive tool in fields seeking to visualize pairwise connectivity.Comment: 4 pages, IEEE NER 201

The reorganization of proper nouns: treatment of proper noun retrieval deficits in an individual with temporal lobe epilepsy

The neural correlates of proper noun retrieval have been investigated through neuroimaging and lesion approaches. Neuroimaging studies investigating proper noun naming in neurologically healthy individuals have demonstrated the importance of the left anterior temporal lobe (ATL) to the integrity of proper noun naming (Gorno-Tempini, 2001; Grabowski, Damasio, & Tranel, 2000; Nakamura, et al., 2000; Tranel, 2009; Tsukiura, et al., 2002), while studies investigating proper noun production in individuals with left temporal lobe lesions have demonstrated a link between left ATL damage and proper noun retrieval deficits (Damasio, Grabowski, Tranel, Hichwa, & Damasio, 1996; Tranel, 2006, 2009; Tranel, Damasio, & Damasio, 1997; Tranel, Feinstein, & Manzel, 2008; Tsukiura, et al., 2002). Though patients with left temporal lobe epilepsy have mostly normal linguistic abilities, they consistently demonstrate deficits in proper noun retrieval (i.e., famous faces and places; Glosser, Salvucci, & Chiaravalloti, 2003; Griffith, et al., 2006; Seidenberg, et al., 2002; Viskontas, McAndrews, & Moscovitch, 2002)

Molecular characterization of microbiota in cerebrospinal fluid from patients with CSF shunt infections using whole genome amplification followed by shotgun sequencing

Understanding the etiology of cerebrospinal fluid (CSF) shunt infections and reinfections requires detailed characterization of associated microorganisms. Traditionally, identification of bacteria present in the CSF has relied on culture methods, but recent studies have used high throughput sequencing of 16S rRNA genes. Here we evaluated the method of shotgun DNA sequencing for its potential to provide additional genomic information. CSF samples were collected from 3 patients near the beginning and end of each of 2 infection episodes. Extracted total DNA was sequenced by: (1) whole genome amplification followed by shotgun sequencing (WGA) and (2) high-throughput sequencing of the 16S rRNA V4 region (16S). Taxonomic assignments of sequences from WGA and 16S were compared with one another and with conventional microbiological cultures. While classification of bacteria was consistent among the 3 approaches, WGA provided additional insights into sample microbiological composition, such as showing relative abundances of microbial versus human DNA, identifying samples of questionable quality, and detecting significant viral load in some samples. One sample yielded sufficient non-human reads to allow assembly of a high-qualit

Cortical Topography of Error-Related High-Frequency Potentials During Erroneous Control in a Continuous Control Brain–Computer Interface

Brain–computer interfaces (BCIs) benefit greatly from performance feedback, but current systems lack automatic, task-independent feedback. Cortical responses elicited from user error have the potential to serve as state-based feedback to BCI decoders. To gain a better understanding of local error potentials, we investigate responsive cortical power underlying error-related potentials (ErrPs) from the human cortex during a one-dimensional center-out BCI task, tracking the topography of high-gamma (70–100 Hz) band power (HBP) specific to BCI error. We measured electrocorticography (ECoG) in three human subjects during dynamic, continuous control over BCI cursor velocity. Subjects used motor imagery and rest to move the cursor toward and subsequently dwell within a target region. We then identified and labeled epochs where the BCI decoder incorrectly moved the cursor in the direction opposite of the subject’s expectations (i.e., BCI error). We found increased HBP in various cortical areas 100–500 ms following BCI error with respect to epochs of correct, intended control. Significant responses were noted in primary somatosensory, motor, premotor, and parietal areas and generally regardless of whether the subject was using motor imagery or rest to move the cursor toward the target. Parts of somatosensory, temporal, and parietal areas exclusively had increased HBP when subjects were using motor imagery. In contrast, only part of the parietal cortex near the angular gyrus exclusively had an increase in HBP during rest. This investigation is, to our knowledge, the first to explore cortical fields changes in the context of continuous control in ECoG BCI. We present topographical changes in HBP characteristic specific to the generation of error. By focusing on continuous control, instead of on discrete control for simple selection, we investigate a more naturalistic setting and provide high ecological validity for characterizing error potentials. Such potentials could be considered as design elements for co-adaptive BCIs in the future as task-independent feedback to the decoder, allowing for more robust and individualized BCIs

Lower levels of Th1 and Th2 cytokines in cerebrospinal fluid (CSF) at the time of initial CSF shunt placement in children are associated with subsequent shunt revision surgeries

OBJECTIVE: We compare cytokine profiles at the time of initial CSF shunt placement between children who required no subsequent shunt revision surgeries and children requiring repeated CSF shunt revision surgeries for CSF shunt failure. We also describe the cytokine profiles across surgical episodes for children who undergo multiple subsequent revision surgeries. METHODS: This pilot study was nested within an ongoing prospective multicenter study collecting CSF samples and clinical data at the time of CSF shunt surgeries since August 2014. We selected cases where CSF was available for children who underwent an initial CSF shunt placement and had no subsequent shunt revision surgeries during \u3e=24 months of follow-up (n = 7); as well as children who underwent an initial CSF shunt placement and then required repeated CSF shunt revision surgeries (n = 3). Levels of 92 human cytokines were measured using the Olink immunoassay and 41 human cytokines were measured using Luminex based bead array on CSF obtained at the time of each child\u27s initial CSF shunt placement and were displayed in heat maps. RESULTS: Qualitatively similar profiles for the majority of cytokines were observed among the patients in each group in both Olink and Luminex assays. Lower levels of MCP-3, CASP-8, CD5, CXCL9, CXCL11, eotaxin, IFN-γ, IL-13, IP-10, and OSM at the time of initial surgery were noted in the children who went on to require multiple surgeries. Pro- and anti-inflammatory cytokines were selected a priori and shown across subsequent revision surgeries for the 3 patients. Cytokine patterns differed between patients, but within a given patient pro-inflammatory and anti-inflammatory cytokines acted in a parallel fashion, with the exception of IL-4. CONCLUSIONS: Heat maps of cytokine levels at the time of initial CSF shunt placement for each child undergoing only a single initial CSF shunt placement and for each child undergoing repeat CSF shunt revision surgeries demonstrated qualitatively similar profiles for the majority of cytokines. Lower levels of MCP-3, CASP-8, CD5, CXCL9, CXCL11, eotaxin, IFN-γ, IL-13, IP-10, and OSM at the time of initial surgery were noted in the children who went on to require multiple surgeries. Better stratification by patient age, etiology, and mechanism of failure is needed to develop a deeper understanding of the mechanism of inflammation in the development of hydrocephalus and response to shunting in children

Superior verbal memory outcome after stereotactic laser amygdalohippocampotomy

Objective: To evaluate declarative memory outcomes in medically refractory epilepsy patients who underwent either a highly selective laser ablation of the amygdalohippocampal complex or a conventional open temporal lobe resection. Methods: Post-operative change scores were examined for verbal memory outcome in epilepsy patients who underwent stereotactic laser amygdalohippocampotomy (SLAH: n = 40) or open resection procedures (n = 40) using both reliable change index (RCI) scores and a 1-SD change metric. Results: Using RCI scores, patients undergoing open resection (12/40, 30.0%) were more likely to decline on verbal memory than those undergoing SLAH (2/40 [5.0%], p = 0.0064, Fisher's exact test). Patients with language dominant procedures were much more likely to experience a significant verbal memory decline following open resection (9/19 [47.4%]) compared to laser ablation (2/19 [10.5%], p = 0.0293, Fisher's exact test). 1 SD verbal memory decline frequently occurred in the open resection sample of language dominant temporal lobe patients with mesial temporal sclerosis (8/10 [80.0%]), although it rarely occurred in such patients after SLAH (2/14, 14.3%) (p = 0.0027, Fisher's exact test). Memory improvement occurred significantly more frequently following SLAH than after open resection. Interpretation: These findings suggest that while verbal memory function can decline after laser ablation of the amygdalohippocampal complex, it is better preserved when compared to open temporal lobe resection. Our findings also highlight that the dominant hippocampus is not uniquely responsible for verbal memory. While this is at odds with our simple and common heuristic of the hippocampus in memory, it supports the findings of non-human primate studies showing that memory depends on broader medial and lateral TL regions