High frequency oscillations as a correlate of visual perception

“NOTICE: this is the author’s version of a work that was accepted for publication in International journal of psychophysiology. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in International journal of psychophysiology , 79, 1, (2011) DOI 10.1016/j.ijpsycho.2010.07.004Peer reviewedPostprin

The oscillatory mechanisms of working memory maintenance

Working memory (WM) is a cognitive process which allows for maintenance of information that is no longer perceived. Although theoretical models have recognized that working memory involves interactions across cell assemblies in multiple brain areas, the exact neural mechanisms which support this process remain unknown. In this thesis I investigate the neural dynamics in the human hippocampus, the ventral, dorsal and frontal cortex as well as the long-range network connectivity across these brain areas to understand how such a distributed network allows for maintenance of various information pieces in WM. The results described here support a model in which working memory relies on dynamic interactions across frequencies (the cross-frequency coupling, CFC) in a distributed network of cortical areas coordinated by the prefrontal cortex. In particular, maintenance of information during a delay period selectively involves the hippocampus, dorsal and ventral visual stream as well as the prefrontal cortex each of which represents different features. The hippocampus contributes to this large network specifically by representing multiple items in working memory. In two independent experiments I observed that the low-frequency activity (a marker of neural inhibition) was linearly reduced across memory loads. Importantly, the hippocampus showed very prominent low-frequency power during maintenance of a single item suggesting that during this condition the neural processing was strongly inhibited. In turn, the broadband gamma activity was linearly increasing as a function of memory load. This pattern of results may be interpreted as reflecting an increased involvement of the hippocampus in representing longer sequences. Importantly, the low-frequency decrease was not static but fluctuated periodically between two different modes. One of the modes was characterized by the load-dependent power decreases and reduced cross-frequency coupling (memory activation mode) whereas the other mode was reflected by the load-independent high levels of power and increased coupling strength (load-independent mode). Crucially, these modes were temporally organized by the phase of an endogenous delta rhythm forming a “hierarchy of oscillations”. This periodicity was essential for the successful performance. Finally, during the memory activation mode the WM capacity limit was inter-individually correlated with the peak frequency change as predicted by the multiplexing model of WM. All these effects were subsequently replicated in an independent dataset. These results suggest that the hippocampus is involved in WM maintenance showing periodic fluctuations between two different oscillatory modes. Parameters of the hippocampal iEEG signal correlate with individual WM capacity, specifically during the memory activation mode. The ventral and dorsal visual stream each contributes to the distributed WM network by representing configuration and spatial information, respectively. Specifically, the alpha power in the ventral visual stream was decreased during maintenance of face identities. In turn, the alpha power was desynchronized in the dorsal visual stream while participants were maintaining face orientations. This shows that the alpha power double dissociates between the feature specific networks in the ventral and dorsal visual stream. These effects are further interpreted as reflecting selective involvement of the dorsal and ventral visual pathway depending on the maintained features. Importantly, each of the visual streams was selectively synchronized with the prefrontal cortex depending on the memory condition and the alpha power. This corroborates a central prediction from the gating by inhibition model which assumes that the increased alpha power serves as the mechanism for gating of information by inhibiting task redundant pathways. Moreover, during maintenance of information the phase of alpha modulated the amplitude of high-frequency activity both in the dorsal and ventral visual stream. Additionally,the low-frequency phase in the prefrontal cortex modulated high-frequency activity both in the dorsal and ventral visual stream. These results suggest that both the dorsal and ventral visual streams are selectively involved during maintenance of distinct features (i.e. face orientation and identity, respectively). They also indicate that the prefrontal cortex selectively gets synchronized with the visual regions depending on the alpha power in that region and the maintained feature. Finally, the activity in the prefrontal cortex influences processing across long distance as evident from changes in the phase synchrony with the visual cortical areas and by modulating gamma power in the visual cortical regions. It is also noted that the ventrolateral prefrontal cortex (vlPFC) contains information regarding abstract rules (i.e. response mapping). In particular, using a multivariate decoding approach I found that the local field potentials recorded from the vlPFC dissociate between different types of responses. At the same time I observed no evidence for the load-dependent or stimulus-specific changes in that brain region. The null effect should be treated with caution. Nevertheless, the current results suggest that the vlPFC may contribute to working memory by processing of abstract rules such as a mapping between the stimulus and the response. Furthermore, I found that the alpha power dependent duty cycle in the vlPFC constrains the duration of the gamma burst which has been suggested as a mechanism for neural inhibition. This finding is important because such a property of the alpha activity has never been observed in a brain region other than the primary sensory cortex. Together, the results presented in this thesis support a model according to which the working memory is a complex and highly dynamic process engaging hierarchies of oscillations across multiple cortical regions. In particular, the hippocampus is important for the multi-item WM. The dorsal and ventral visual streams are relevant for distinct visual features. Finally, the prefrontal cortex represents abstract rules and influences processing in other cortical regions likely providing a top down control over these regions

Decoding the dynamic representation of facial expressions of emotion in explicit and incidental tasks

Faces transmit a wealth of important social signals. While previous studies have elucidated the network of cortical regions important for perception of facial expression, and the associated temporal components such as the P100, N170 and EPN, it is still unclear how task constraints may shape the representation of facial expression (or other face categories) in these networks. In the present experiment, we used Multivariate Pattern Analysis (MVPA) with EEG to investigate the neural information available across time about two important face categories (expression and identity) when those categories are either perceived under explicit (e.g. decoding facial expression category from the EEG when task is on expression) or incidental task contexts (e.g. decoding facial expression category from the EEG when task is on identity). Decoding of both face categories, across both task contexts, peaked in time-windows spanning 91–170 ms (across posterior electrodes). Peak decoding of expression, however, was not affected by task context whereas peak decoding of identity was significantly reduced under incidental processing conditions. In addition, errors in EEG decoding correlated with errors in behavioral categorization under explicit processing for both expression and identity, but only with incidental decoding of expression. Furthermore, decoding time-courses and the spatial pattern of informative electrodes showed consistently better decoding of identity under explicit conditions at later-time periods, with weak evidence for similar effects for decoding of expression at isolated time-windows. Taken together, these results reveal differences and commonalities in the processing of face categories under explicit Vs incidental task contexts and suggest that facial expressions are processed to a richer degree under incidental processing conditions, consistent with prior work indicating the relative automaticity by which emotion is processed. Our work further demonstrates the utility in applying multivariate decoding analyses to EEG for revealing the dynamics of face perception

Pinging the brain with visual impulses reveals electrically active, not activity-silent, working memories

Persistently active neurons during mnemonic periods have been regarded as the mechanism underlying working memory maintenance. Alternatively, neuronal networks could instead store memories in fast synaptic changes, thus avoiding the biological cost of maintaining an active code through persistent neuronal firing. Such "activity-silent" codes have been proposed for specific conditions in which memories are maintained in a nonprioritized state, as for unattended but still relevant short-term memories. A hallmark of this "activity-silent" code is that these memories can be reactivated from silent, synaptic traces. Evidence for "activity-silent" working memory storage has come from human electroencephalography (EEG), in particular from the emergence of decodability (EEG reactivations) induced by visual impulses (termed pinging) during otherwise "silent" periods. Here, we reanalyze EEG data from such pinging studies. We find that the originally reported absence of memory decoding reflects weak statistical power, as decoding is possible based on more powered analyses or reanalysis using alpha power instead of raw voltage. This reveals that visual pinging EEG "reactivations" occur in the presence of an electrically active, not silent, code for unattended memories in these data. This crucial change in the evidence provided by this dataset prompts a reinterpretation of the mechanisms of EEG reactivations. We provide 2 possible explanations backed by computational models, and we discuss the relationship with TMS-induced EEG reactivations

Ongoing neural oscillations influence behavior and sensory representations by suppressing neuronal excitability

The ability to process and respond to external input is critical for adaptive behavior. Why, then, do neural and behavioral responses vary across repeated presentations of the same sensory input? Ongoing fluctuations of neuronal excitability are currently hypothesized to underlie the trial-by-trial variability in sensory processing. To test this, we capitalized on intracranial electrophysiology in neurosurgical patients performing an auditory discrimination task with visual cues: specifically, we examined the interaction between prestimulus alpha oscillations, excitability, task performance, and decoded neural stimulus representations. We found that strong prestimulus oscillations in the alpha+ band (i.e., alpha and neighboring frequencies), rather than the aperiodic signal, correlated with a low excitability state, indexed by reduced broadband high-frequency activity. This state was related to slower reaction times and reduced neural stimulus encoding strength. We propose that the alpha+ rhythm modulates excitability, thereby resulting in variability in behavior and sensory representations despite identical input

Using Brain–Computer Interfaces and Brain-State Dependent Stimulation as Tools in Cognitive Neuroscience

Large efforts are currently being made to develop and improve online analysis of brain activity which can be used, e.g., for brain–computer interfacing (BCI). A BCI allows a subject to control a device by willfully changing his/her own brain activity. BCI therefore holds the promise as a tool for aiding the disabled and for augmenting human performance. While technical developments obviously are important, we will here argue that new insight gained from cognitive neuroscience can be used to identify signatures of neural activation which reliably can be modulated by the subject at will. This review will focus mainly on oscillatory activity in the alpha band which is strongly modulated by changes in covert attention. Besides developing BCIs for their traditional purpose, they might also be used as a research tool for cognitive neuroscience. There is currently a strong interest in how brain-state fluctuations impact cognition. These state fluctuations are partly reflected by ongoing oscillatory activity. The functional role of the brain state can be investigated by introducing stimuli in real-time to subjects depending on the actual state of the brain. This principle of brain-state dependent stimulation may also be used as a practical tool for augmenting human behavior. In conclusion, new approaches based on online analysis of ongoing brain activity are currently in rapid development. These approaches are amongst others informed by new insight gained from electroencephalography/magnetoencephalography studies in cognitive neuroscience and hold the promise of providing new ways for investigating the brain at work

Revealing hidden states in visual working memory using electroencephalography

It is often assumed that information in visual working memory (vWM) is maintained via persistent activity. However, recent evidence indicates that information in vWM could be maintained in an effectively "activity-silent" neural state. Silent vWM is consistent with recent cognitive and neural models, but poses an important experimental problem: how can we study these silent states using conventional measures of brain activity? We propose a novel approach that is analogous to echolocation: using a high-contrast visual stimulus, it may be possible to drive brain activity during vWM maintenance and measure the vWM-dependent impulse response. We recorded electroencephalography (EEG) while participants performed a vWM task in which a randomly oriented grating was remembered. Crucially, a high-contrast, task-irrelevant stimulus was shown in the maintenance period in half of the trials. The electrophysiological response from posterior channels was used to decode the orientations of the gratings. While orientations could be decoded during and shortly after stimulus presentation, decoding accuracy dropped back close to baseline in the delay. However, the visual evoked response from the task-irrelevant stimulus resulted in a clear re-emergence in decodability. This result provides important proof-of-concept for a promising and relatively simple approach to decode "activity-silent" vWM content using non-invasive EEG

Inimaju arvutuslikke protsesside mõistmine masinõpe mudelite tõlgendamise kaudu. Andmepõhine lähenemine arvutuslikku neuroteadusesse

Modelleerimine on inimkonna põline viis keerulistest nähtustest arusaamiseks. Planeetide liikumise mudel, gravitatsiooni mudel ja osakestefüüsika standardmudel on näited selle lähenemise edukusest. Neuroteaduses on olemas kaks viisi mudelite loomiseks: traditsiooniline hüpoteesipõhine lähenemine, mille puhul kõigepealt mudel sõnastatakse ja alles siis valideeritakse andmete peal; ja uuem andmepõhine lähenemine, mis toetub masinõpele, et sõnastada mudeleid automaatselt. Hüpoteesipõhine viis annab täieliku mõistmise sellest, kuidas mudel töötab, aga nõuab aega, kuna iga hüpotees peab olema sõnastatud ja valideeritud käsitsi. Andmepõhine lähenemine toetub ainult andmetele ja arvutuslikele ressurssidele mudelite otsimisel, aga ei seleta kuidas täpselt mudel jõuab oma tulemusteni. Me väidame, et neuroandmestike suur hulk ja nende mahu kiire kasv nõuab andmepõhise lähenemise laiemat kasutuselevõttu neuroteaduses, nihkes uurija rolli mudelite tööprintsiipide tõlgendamisele. Doktoritöö koosneb kolmest näitest neuroteaduse teadmisi avastamisest masinõppe tõlgendamismeetodeid kasutades. Esimeses uuringus tõlgendatava mudeli abiga me kirjeldame millised ajas muutuvad sageduskomponendid iseloomustavad inimese ajusignaali visuaalsete objektide tuvastamise ülesande puhul. Teises uuringus võrdleme omavahel signaale inimese aju ventraalses piirkonnas ja konvolutsiooniliste tehisnärvivõrkude aktivatsioone erinevates kihtides. Säärane võrdlus võimaldas meil kinnitada hüpoteesi, et mõlemad süsteemid kasutavad hierarhilist struktuuri. Viimane näide kasutab topoloogiat säilitavat mõõtmelisuse vähendamise ja visualiseerimise meetodit, et näha, millised ajusignaalid ja mõtteseisundid on üksteisele sarnased. Viimased tulemused masinõppes ja tehisintellektis näitasid et mõned mehhanismid meie ajus on sarnased mehhanismidega, milleni jõuavad õppimise käigus masinõppe algoritmid. Oma tööga me rõhutame masinõppe mudelite tõlgendamise tähtsust selliste mehhanismide avastamiseks.Building a model of a complex phenomenon is an ancient way of gaining knowledge and understanding of the reality around us. Models of planetary motion, gravity, particle physics are examples of this approach. In neuroscience, there are two ways of coming up with explanations of reality: a traditional hypothesis-driven approach, where a model is first formulated and then tested using the data, and a more recent data-driven approach, that relies on machine learning to generate models automatically. Hypothesis-driven approach provides full understanding of the model, but is time-consuming as each model has to be conceived and tested manually. Data-driven approach requires only the data and computational resources to sift through potential models, saving time, but leaving the resulting model itself to be a black box. Given the growing amount of neural data, we argue in favor of a more widespread adoption of the data-driven approach, reallocating part of the human effort from manual modeling. The thesis is based on three examples of how interpretation of machine-learned models leads to neuroscientific insights on three different levels of neural organization. Our first interpretable model is used to characterize neural dynamics of localized neural activity during the task of visual perceptual categorization. Next, we compare the activity of human visual system with the activity of a convolutional neural network, revealing explanations about the functional organization of human visual cortex. Lastly, we use dimensionality reduction and visualization techniques to understand relative organization of mental concepts within a subject's mental state space and apply it in the context of brain-computer interfaces. Recent results in neuroscience and AI show similarities between the mechanisms of both systems. This fact endorses the relevance of our approach: interpreting the mechanisms employed by machine learning models can shed light on the mechanisms employed by our brainhttps://www.ester.ee/record=b536057

It's about Time

The purpose of this review/opinion paper is to argue that human cognitive neuroscience has focused too little attention on how the brain may use time and time-based coding schemes to represent, process, and transfer information within and across brain regions. Instead, the majority of cognitive neuroscience studies rest on the assumption of functional localization. Although the functional localization approach has brought us a long way towards a basic characterization of brain functional organization, there are methodological and theoretical limitations of this approach. Further advances in our understanding of neurocognitive function may come from examining how the brain performs computations and forms transient functional neural networks using the rich multi-dimensional information available in time. This approach rests on the assumption that information is coded precisely in time but distributed in space; therefore, measures of rapid neuroelectrophysiological dynamics may provide insights into brain function that cannot be revealed using localization-based approaches and assumptions. Space is not an irrelevant dimension for brain organization; rather, a more complete understanding of how brain dynamics lead to behavior dynamics must incorporate how the brain uses time-based coding and processing schemes