Unsupervised decoding of long-term, naturalistic human neural recordings with automated video and audio annotations

Fully automated decoding of human activities and intentions from direct neural recordings is a tantalizing challenge in brain-computer interfacing. Most ongoing efforts have focused on training decoders on specific, stereotyped tasks in laboratory settings. Implementing brain-computer interfaces (BCIs) in natural settings requires adaptive strategies and scalable algorithms that require minimal supervision. Here we propose an unsupervised approach to decoding neural states from human brain recordings acquired in a naturalistic context. We demonstrate our approach on continuous long-term electrocorticographic (ECoG) data recorded over many days from the brain surface of subjects in a hospital room, with simultaneous audio and video recordings. We first discovered clusters in high-dimensional ECoG recordings and then annotated coherent clusters using speech and movement labels extracted automatically from audio and video recordings. To our knowledge, this represents the first time techniques from computer vision and speech processing have been used for natural ECoG decoding. Our results show that our unsupervised approach can discover distinct behaviors from ECoG data, including moving, speaking and resting. We verify the accuracy of our approach by comparing to manual annotations. Projecting the discovered cluster centers back onto the brain, this technique opens the door to automated functional brain mapping in natural settings

Independent component analysis of interictal fMRI in focal epilepsy: comparison with general linear model-based EEG-correlated fMRI

The general linear model (GLM) has been used to analyze simultaneous EEG–fMRI to reveal BOLD changes linked to interictal epileptic discharges (IED) identified on scalp EEG. This approach is ineffective when IED are not evident in the EEG. Data-driven fMRI analysis techniques that do not require an EEG derived model may offer a solution in these circumstances. We compared the findings of independent components analysis (ICA) and EEG-based GLM analyses of fMRI data from eight patients with focal epilepsy. Spatial ICA was used to extract independent components (IC) which were automatically classified as either BOLD-related, motion artefacts, EPI-susceptibility artefacts, large blood vessels, noise at high spatial or temporal frequency. The classifier reduced the number of candidate IC by 78%, with an average of 16 BOLD-related IC. Concordance between the ICA and GLM-derived results was assessed based on spatio-temporal criteria. In each patient, one of the IC satisfied the criteria to correspond to IED-based GLM result. The remaining IC were consistent with BOLD patterns of spontaneous brain activity and may include epileptic activity that was not evident on the scalp EEG. In conclusion, ICA of fMRI is capable of revealing areas of epileptic activity in patients with focal epilepsy and may be useful for the analysis of EEG–fMRI data in which abnormalities are not apparent on scalp EEG

MEG sensor and source measures of visually induced gamma-band oscillations are highly reliable

High frequency brain oscillations are associated with numerous cognitive and behavioral processes. Non-invasive measurements using electro-/magnetoencephalography (EEG/MEG) have revealed that high frequency neural signals are heritable and manifest changes with age as well as in neuropsychiatric illnesses. Despite the extensive use of EEG/MEG-measured neural oscillations in basic and clinical research, studies demonstrating test–retest reliability of power and frequency measures of neural signals remain scarce. Here, we evaluated the test–retest reliability of visually induced gamma (30–100 Hz) oscillations derived from sensor and source signals acquired over two MEG sessions. The study required participants (N = 13) to detect the randomly occurring stimulus acceleration while viewing a moving concentric grating. Sensor and source MEG measures of gamma-band activity yielded comparably strong reliability (average intraclass correlation, ICC = 0.861). Peak stimulus-induced gamma frequency (53–72 Hz) yielded the highest measures of stability (ICCsensor = 0.940; ICCsource = 0.966) followed by spectral signal change (ICCsensor = 0.890; ICCsource = 0.893) and peak frequency bandwidth (ICCsensor = 0.856; ICCsource = 0.622). Furthermore, source-reconstruction significantly improved signal-to-noise for spectral amplitude of gamma activity compared to sensor estimates. Our assessments highlight that both sensor and source derived estimates of visually induced gamma-band oscillations from MEG signals are characterized by high test–retest reliability, with source derived oscillatory measures conferring an improvement in the stability of peak-frequency estimates. Importantly, our finding of high test–retest reliability supports the feasibility of pharma-MEG studies and longitudinal aging or clinical studies

Deep Cellular Recurrent Neural Architecture for Efficient Multidimensional Time-Series Data Processing

Efficient processing of time series data is a fundamental yet challenging problem in pattern recognition. Though recent developments in machine learning and deep learning have enabled remarkable improvements in processing large scale datasets in many application domains, most are designed and regulated to handle inputs that are static in time. Many real-world data, such as in biomedical, surveillance and security, financial, manufacturing and engineering applications, are rarely static in time, and demand models able to recognize patterns in both space and time. Current machine learning (ML) and deep learning (DL) models adapted for time series processing tend to grow in complexity and size to accommodate the additional dimensionality of time. Specifically, the biologically inspired learning based models known as artificial neural networks that have shown extraordinary success in pattern recognition, tend to grow prohibitively large and cumbersome in the presence of large scale multi-dimensional time series biomedical data such as EEG. Consequently, this work aims to develop representative ML and DL models for robust and efficient large scale time series processing. First, we design a novel ML pipeline with efficient feature engineering to process a large scale multi-channel scalp EEG dataset for automated detection of epileptic seizures. With the use of a sophisticated yet computationally efficient time-frequency analysis technique known as harmonic wavelet packet transform and an efficient self-similarity computation based on fractal dimension, we achieve state-of-the-art performance for automated seizure detection in EEG data. Subsequently, we investigate the development of a novel efficient deep recurrent learning model for large scale time series processing. For this, we first study the functionality and training of a biologically inspired neural network architecture known as cellular simultaneous recurrent neural network (CSRN). We obtain a generalization of this network for multiple topological image processing tasks and investigate the learning efficacy of the complex cellular architecture using several state-of-the-art training methods. Finally, we develop a novel deep cellular recurrent neural network (CDRNN) architecture based on the biologically inspired distributed processing used in CSRN for processing time series data. The proposed DCRNN leverages the cellular recurrent architecture to promote extensive weight sharing and efficient, individualized, synchronous processing of multi-source time series data. Experiments on a large scale multi-channel scalp EEG, and a machine fault detection dataset show that the proposed DCRNN offers state-of-the-art recognition performance while using substantially fewer trainable recurrent units