Learning hierarchical sequence representations across human cortex and hippocampus

Sensory input arrives in continuous sequences that humans experience as segmented units, e.g., words and events. The brain’s ability to discover regularities is called statistical learning. Structure can be represented at multiple levels, including transitional probabilities, ordinal position, and identity of units. To investigate sequence encoding in cortex and hippocampus, we recorded from intracranial electrodes in human subjects as they were exposed to auditory and visual sequences containing temporal regularities. We find neural tracking of regularities within minutes, with characteristic profiles across brain areas. Early processing tracked lower-level features (e.g., syllables) and learned units (e.g., words), while later processing tracked only learned units. Learning rapidly shaped neural representations, with a gradient of complexity from early brain areas encoding transitional probability, to associative regions and hippocampus encoding ordinal position and identity of units. These findings indicate the existence of multiple, parallel computational systems for sequence learning across hierarchically organized cortico-hippocampal circuits

Deep phenotyping of the unselected COPSAC2010 birth cohort study

BACKGROUND: We hypothesize that perinatal exposures, in particular the human microbiome and maternal nutrition during pregnancy, interact with the genetic predisposition to cause an abnormal immune modulation in early life towards a trajectory to chronic inflammatory diseases such as asthma and others. OBJECTIVE: The aim of this study is to explore these interactions by conducting a longitudinal study in an unselected cohort of pregnant women and their offspring with emphasis on deep clinical phenotyping, exposure assessment, and biobanking. Exposure assessments focus on the human microbiome. Nutritional intervention during pregnancy in randomized controlled trials are included in the study to prevent disease and to be able to establish causal relationships. METHODS: Pregnant women from eastern Denmark were invited during 2008–2010 to a novel unselected ‘COPSAC(2010)’ cohort. The women visited the clinic during pregnancy weeks 24 and 36. Their children were followed at the clinic with deep phenotyping and collection of biological samples at nine regular visits until the age of 3 and at acute symptoms. Randomized controlled trials of high‐dose vitamin D and fish oil supplements were conducted during pregnancy, and a trial of azithromycin for acute lung symptoms was conducted in the children with recurrent wheeze. RESULTS: Seven hundred and thirty‐eight mothers were recruited from week 24 of gestation, and 700 of their children were included in the birth cohort. The cohort has an over‐representation of atopic parents. The participant satisfaction was high and the adherence equally high with 685 children (98%) attending the 1 year clinic visit and 667 children (95%) attending the 2 year clinic visit. CONCLUSIONS: The COPSAC(2010) birth cohort study provides longitudinal clinical follow‐up with highly specific end‐points, exposure assessments, and biobanking. The cohort has a high adherence rate promising strong data to elucidate the interaction between genomics and the exposome in perinatal life leading to lifestyle‐related chronic inflammatory disorders such as asthma

Shared computational principles for language processing in humans and deep language models

Departing from traditional linguistic models, advances in deep learning have resulted in a new type of predictive (autoregressive) deep language models (DLMs). Using a self-supervised next-word prediction task, these models generate appropriate linguistic responses in a given context. In the current study, nine participants listened to a 30-min podcast while their brain responses were recorded using electrocorticography (ECoG). We provide empirical evidence that the human brain and autoregressive DLMs share three fundamental computational principles as they process the same natural narrative: (1) both are engaged in continuous next-word prediction before word onset; (2) both match their pre-onset predictions to the incoming word to calculate post-onset surprise; (3) both rely on contextual embeddings to represent words in natural contexts. Together, our findings suggest that autoregressive DLMs provide a new and biologically feasible computational framework for studying the neural basis of language

Reconstructing Speech from Human Auditory Cortex

Direct brain recordings from neurosurgical patients listening to speech reveal that the acoustic speech signals can be reconstructed from neural activity in auditory cortex

The Relative Contribution of High-Gamma Linguistic Processing Stages of Word Production, and Motor Imagery of Articulation in Class Separability of Covert Speech Tasks in EEG Data

Word production begins with high-Gamma automatic linguistic processing functions followed by speech motor planning and articulation. Phonetic properties are processed in both linguistic and motor stages of word production. Four phonetically dissimilar phonemic structures “BA”, “FO”, “LE”, and “RY” were chosen as covert speech tasks. Ten neurologically healthy volunteers with the age range of 21–33 participated in this experiment. Participants were asked to covertly speak a phonemic structure when they heard an auditory cue. EEG was recorded with 64 electrodes at 2048 samples/s. Initially, one-second trials were used, which contained linguistic and motor imagery activities. The four-class true positive rate was calculated. In the next stage, 312 ms trials were used to exclude covert articulation from analysis. By eliminating the covert articulation stage, the four-class grand average classification accuracy dropped from 96.4% to 94.5%. The most valuable features emerge after Auditory cue recognition (~100 ms post onset), and within the 70–128 Hz frequency range. The most significant identified brain regions were the Prefrontal Cortex (linked to stimulus driven executive control), Wernicke’s area (linked to Phonological code retrieval), the right IFG, and Broca’s area (linked to syllabification). Alpha and Beta band oscillations associated with motor imagery do not contain enough information to fully reflect the complexity of speech movements. Over 90% of the most class-dependent features were in the 30-128 Hz range, even during the covert articulation stage. As a result, compared to linguistic functions, the contribution of motor imagery of articulation in class separability of covert speech tasks from EEG data is negligible

Modeling the temporal dynamics of neural responses in human visual cortex

Cortical responses to visual stimuli exhibit complex temporal dynamics, including sub-additive temporal summation, response reduction with repeated or sustained stimuli (adaptation), and slower dynamics at low contrast. Multiple computational models have been proposed to account for these dynamics in several measurement domains, including single-cell recordings, psychophysics, and fMRI. It is challenging to compare these models because there are differences in model form, test stimuli, and instrument. Here we present a new dataset that is well-suited to compare models of neural temporal dynamics. The dataset is from electrocorticographic (ECoG) recordings of human visual cortex, which measures cortical neural population responses with high spatial and temporal precision. The stimuli were large, static contrast patterns and varied systematically in contrast, duration, and inter-stimulus interval (ISI). Time-varying broadband responses were computed using the power envelope of the band-pass filtered voltage time course (50-170 Hz) recorded from a total of 126 electrodes in ten epilepsy patients, covering earlier (V1-V4) and higher-order (LO, TO, IPS) retinotopic maps. In all visual regions, the ECoG broadband responses show several non-linear features: peak response amplitude saturates with high contrast and long stimulus durations; response latency decreases with increasing contrast; and the response to a second stimulus is suppressed for short ISIs and recovers for longer ISIs. These features were well predicted by a computational model (Zhou, Benson, Kay and Winawer, 2019) comprised of a small set of canonical neuronal operations: linear filtering, rectification, exponentiation, and a delayed divisive gain control. These results demonstrate that a simple computational model comprised of canonical neuronal computations captures a wide range of temporal and contrast-dependent neuronal dynamics at millisecond resolution. Finally, we present a software repository that implements models of temporal dynamics in a modular fashion, enabling the comparison of many models fit to the same data and analyzed with the same methods