Tapahtumasegmentaation aivovasteet hippokampuksessa ja aivokuorella äänitarinan kuuntelun aikana

Tapahtumasegmentaatio jäsentää sekä arkista kokemustamme että muistiamme. Parhaillaan meneillään olevan tapahtuman hahmotus ja prosessointi tapahtuu todennäköisesti aivokuorella, mutta ilman toimivaa hippokampusta tilanteesta ei voi syntyä pysyvää muistoa. On olennainen kysymys, missä kohtaa ja miten hippokampus osallistuu tapahtumien prosessointiin ja mieleen painamiseen. Aiemmin on magneettikuvaustutkimuksin osoitettu, että hippokampus reagoi tapahtumien välisiin rajoihin aktivaatiopiikein. On ehdotettu, että ne ilmentäisivät aistimodaliteetista riippumattoman tason prosessia, jossa hippokampus kokoaa yhteen ja vahvistaa koetun tilanteen kokonaisrepresentaation, jotta se voidaan painaa muistiin. Aiemmat tutkimukset on kuitenkin toteutettu yksinomaan audiovisuaalisilla ärsykkeillä, ja koska hippokampuksen tiedetään osallistuvan myös visuaaliseen prosessointiin, ei ole täysin selvää, etteivätkö havaitut aktivaatiot voisi selittyä alemman, aistitietoa käsittelevän tason prosesseilla. Tämän kysymyksen ratkaisemiseksi tässä tutkimuksessa selvitettiin reagoiko hippokampus tapahtumarajoihin puhtaasti auditiivisessa ärsykkeessä. Ärsykkeenä oli 71-minuuttinen tarinallinen äänikirja, jonka osallistujat kuuntelivat passiivisesti fMRI-rekisteröinnin aikana, ja jonka tapahtumarajat määriteltiin kokeellisesti erillisen koehenkilöryhmän avulla. Aivokuvausaineisto analysoitiin aivoalueittain sekä hippokampuksesta että eksploratiivisesti myös kaikilta aivokuoren alueilta. Hippokampuksen havaittiin reagoivan tapahtumarajoihin aktivaatiopiikein. Aivokuorella voimakkaasti reagoivia alueita olivat mm. posteriorinen mediaalinen aivokuori, ventromediaalinen prefrontaalialue, parahippokampaalinen poimu sekä etummainen pihtipoimu. Monien näistä alueista uskotaan osallistuvan meneillään olevan tapahtuman mallintamiseen ja hahmottamiseen, ja osa mahdollisesti osallistuu huomion siirtämiseen sisäisen ja ulkoisen välillä. Etummaisen pihtipoimun tiedetään osallistuvan odotusten ja havaintojen välisten konfliktien monitorointiin, mikä saattaisi tukea teoriaa, jonka mukaan segmentaatio olisi riippuvaista havaituista ennustevirheistä. Tätä ei kuitenkaan tämän tutkimuksen perusteella voida varmasti päätellä, vaan asiaa tulisi tutkia tarkemmin. Tämän tutkimuksen tulokset tukevat näkemystä, jonka mukaan hippokampuksen lisääntynyt toiminta tapahtumarajoilla liittyy korkean tason abstraktiin segmentaatioon ja mahdollisesti episodisen muiston luomiseen. Tämä prosessi mahdollisesti tapahtuu yhteistyössä aivokuoren aktiivisten alueiden kanssa, mutta kausaaliset suhteet ja informaation kulku näiden alueiden välillä on selvitettävä myöhemmissä tutkimuksissa.Event segmentation structures our experience as well as our memories. The representation of the currently ongoing event is likely dependent on a network of cortical areas, but the ability to retain a memory of the event requires an intact hippocampus. It is thus a relevant question how and when this hippocampal episodic encoding happens. It has previously been shown that the hippocampus is sensitive to event boundaries and responds to them with transient fMRI activation peaks. It has been proposed that these hippocampal end-of-event activations represent a high-level, modality-independent process of sharpening or “printing out” of the memory trace of the situation. However, the studies reporting hippocampal peaks have been conducted on audio-visual stimuli, so it is unclear whether these results generalise to narratives without a visual component, as the hippocampus is known to support visual processing as well as episodic encoding. In this study I aim to answer this question by analysing fMRI data from participants experiencing a purely auditory narrative. The stimulus was a 71-minute-long audio book, and it was segmented behaviourally by a separate group of participants with a naïve intuitive segmentation paradigm. The data was analysed with a region of interest (ROI) analysis in the hippocampus, as well as in an exploratory manner on all areas from a cortical atlas. The hippocampus was found to respond significantly to event boundaries in the story. Strong responses were also found in areas of the posterior medial cortex (PMC), as well as in ventromedial prefrontal cortex (vmPFC), parahippocampal gyrus, anterior cingulate (ACC) and the insula. Many of these are known to be involved in representing the event model, and some with switching between internal and external processing modes. ACC in particular is known to be involved in conflict monitoring – this might link with the proposal that segmentation in general is driven by prediction error and would merit further study. I conclude that the hippocampus does detect and respond to event boundaries in a naturalistic auditory narrative, which is in line with the “print out” hypothesis and implies that these activations are related to domain-general episodic encoding. The increased hippocampal processing is likely to happen in collaboration with cortical areas involved in signalling change and representing the working event model. However, the causal connections between these areas during the boundary-related processing cascade needs to be elaborated in future studies

Naturalistic Sentence Comprehension in the Brain

The cognitive neuroscience of language relies largely on controlled experiments that are different from the everyday situations in which we use language. This review describes an approach that studies specific aspects of sentence comprehension in the brain using data collected while participants perform an everyday task, such as listening to a story. The approach uses ‘neuro‐computational’ models that are based on linguistic and psycholinguistic theories. These models quantify how a specific computation, such as identifying a syntactic constituent, might be carried out by a neural circuit word‐by‐word. Model predictions are tested for their statistical fit with measured brain data. The paper discusses three applications of this approach: (i) to probe the location and timing of linguistic processing in the brain without requiring unnatural tasks and stimuli, (ii) to test theoretical hypotheses by comparing the fits of different models to naturalistic data, and (iii) to study neural mechanisms for language processing in populations that are poorly served by traditional methods.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/133583/1/lnc312198.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/133583/2/lnc312198_am.pd

What Physiological Changes and Cerebral Traces Tell Us about Adhesion to Fiction During Theater-Watching?

Live theater is typically designed to alter the state of mind of the audience. Indeed, the perceptual inputs issuing from a live theatrical performance are intended to represent something else, and the actions, emphasized by the writing and staging, are the key prompting the adhesion of viewers to fiction, i.e., their belief that it is real. This phenomenon raises the issue of the cognitive processes governing access to a fictional reality during live theater and of their cerebral underpinnings. To get insight into the physiological substrates of adhesion we recreated the peculiar context of watching live drama in a functional magnetic resonance imaging (fMRI) experiment, with simultaneous recording of heart activity. The instants of adhesion were defined as the co-occurrence of theatrical events determined a priori by the stage director and the spectators’ offline reports of moments when fiction acted as reality. These data served to specify, for each spectator, individual fMRI time-series, used in a random-effect group analysis to define the pattern of brain response to theatrical events. The changes in this pattern related to subjects’ adhesion to fiction, were investigated using a region of interest analysis. The results showed that adhesion to theatrical events correlated with increased activity in the left BA47 and posterior superior temporal sulcus, together with a decrease in dynamic heart rate variability, leading us to discuss the hypothesis of subtle changes in the subjects’ state of awareness, enabling them to mentally dissociate physical and mental (drama-viewing) experiences, to account for the phenomenon of adhesion to dramatic fiction

Narrative comprehension beyond language : Common brain networks activated by a movie and its script

Narratives surround us in our everyday life in different forms. In the sensory brain areas, the processing of narratives is dependent on the media of presentation, be that in audiovisual or written form. However, little is known of the brain areas that process complex narrative content mediated by various forms. To isolate these regions, we looked for the functional networks reacting in a similar manner to the same narrative content despite different media of presentation. We collected 3-T fMRI whole brain data from 31 healthy human adults during two separate runs when they were either viewing a movie or reading its screenplay text. The independent component analysis (ICA) was used to separate 40 components. By correlating the components' time-courses between the two different media conditions, we could isolate 5 functional networks that particularly related to the same narrative content. These TOP-5 components with the highest correlation covered fronto-temporal, parietal, and occipital areas with no major involvement of primary visual or auditory cortices. Interestingly, the top-ranked network with highest modality-invariance also correlated negatively with the dialogue predictor, thus pinpointing that narrative comprehension entails processes that are not language-reliant. In summary, our novel experiment design provided new insight into narrative comprehension networks across modalities.Peer reviewe

Autonomic responses to emotional linguistic stimuli and amplitude of low-frequency fluctuations predict outcome after severe brain injury

An accurate prognosis on the outcome of brain-injured patients with disorders of consciousness (DOC) remains a significant challenge, especially in the acute stage. In this study, we applied a multiple-technique approach to provide accurate predictions on functional outcome after 6 months in 15 acute DOC patients. Electrophysiological correlates of implicit cognitive processing of verbal stimuli and data-driven voxel-wise resting-state fMRI signals, such as the fractional amplitude of low-frequency fluctuations (fALFF), were employed. Event-related electrodermal activity, an index of autonomic activation, was recorded in response to emotional words and pseudo-words at baseline (T0). On the same day, patients also underwent a resting-state fMRI scan. Six months later (T1), patients were classified as outcome-negative and outcome-positive using a standard functional outcome scale. We then revisited the baseline measures to test their predictive power for the functional outcome measured at T1. We found that only outcome-positive patients had an earlier, higher autonomic response for words compared to pseudo-words, a pattern similar to that of healthy awake controls. Furthermore, DOC patients showed reduced fALFF in the posterior cingulate cortex (PCC), a brain region that contributes to autonomic regulation and awareness. The event-related electrodermal marker of residual cognitive functioning was found to have a significant correlation with residual local neuronal activity in the PCC. We propose that a residual autonomic response to cognitively salient stimuli, together with a preserved resting-state activity in the PCC, can provide a useful prognostic index in acute DOC

Neural correlates of coherence-break detection during reading of narratives

Article / Letter to editorInstituut Pedagogische Wetenschappe

The Neural Mechanisms Supporting Structure and Inter-Brain Connectivity In Natural Conversation

Conversation is the height of human communication and social interaction, yet little is known about the neural mechanisms supporting it. To date, there have been no ecologically valid neuroimaging studies of conversation, and for good reason. Until recently, imaging techniques were hindered by artifact related to speech production. Now that we can circumvent this problem, I attempt to uncover the neural correlates of multiple aspects of conversation, including coordinating speaker change, the effect of conversation type (e.g. cooperative or argumentative) on inter-brain coupling, and the relationship between this coupling and social coherence. Pairs of individuals underwent simultaneous fMRI brain scans while they engaged in a series of unscripted conversations, for a total of 40 pairs (80 individuals). The first two studies in this dissertation lay a foundation by outlining brain regions supporting comprehension and production in both narrative and conversation - two aspects of discourse level communication. The subsequent studies focus on two unique features of conversation: alternating turns-at-talk and establishing inter-brain coherence through speech. The results show that at the moment of speaker change, both people are engaging attentional and mentalizing systems - which likely support orienting toward implicit cues signaling speaker change as well as anticipating the other person's intention to either begin or end his turn. Four networks were identified that are significantly predicted by a novel measure of social coherence; they include the posterior parietal cortex, medial prefrontal cortex, and right angular gyrus. Taken together, the findings reveal that natural conversation relies on multiple cognitive networks besides language to coordinate or enhance social interaction. &#8195

Computational modelling of neural mechanisms underlying natural speech perception

Humans are highly skilled at the analysis of complex auditory scenes. In particular, the human auditory system is characterized by incredible robustness to noise and can nearly effortlessly isolate the voice of a specific talker from even the busiest of mixtures. However, neural mechanisms underlying these remarkable properties remain poorly understood. This is mainly due to the inherent complexity of speech signals and multi-stage, intricate processing performed in the human auditory system. Understanding these neural mechanisms underlying speech perception is of interest for clinical practice, brain-computer interfacing and automatic speech processing systems. In this thesis, we developed computational models characterizing neural speech processing across different stages of the human auditory pathways. In particular, we studied the active role of slow cortical oscillations in speech-in-noise comprehension through a spiking neural network model for encoding spoken sentences. The neural dynamics of the model during noisy speech encoding reflected speech comprehension of young, normal-hearing adults. The proposed theoretical model was validated by predicting the effects of non-invasive brain stimulation on speech comprehension in an experimental study involving a cohort of volunteers. Moreover, we developed a modelling framework for detecting the early, high-frequency neural response to the uninterrupted speech in non-invasive neural recordings. We applied the method to investigate top-down modulation of this response by the listener's selective attention and linguistic properties of different words from a spoken narrative. We found that in both cases, the detected responses of predominantly subcortical origin were significantly modulated, which supports the functional role of feedback, between higher- and lower levels stages of the auditory pathways, in speech perception. The proposed computational models shed light on some of the poorly understood neural mechanisms underlying speech perception. The developed methods can be readily employed in future studies involving a range of experimental paradigms beyond these considered in this thesis.Open Acces

Language beyond the language system:Dorsal visuospatial pathways support processing of demonstratives and spatial language during naturalistic fast fMRI

Spatial demonstratives are powerful linguistic tools used to establish joint attention. Identifying the meaning of semantically underspecified expressions like “this one” hinges on the integration of linguistic and visual cues, attentional orienting and pragmatic inference. This synergy between language and extralinguistic cognition is pivotal to language comprehension in general, but especially prominent in demonstratives. In this study, we aimed to elucidate which neural architectures enable this intertwining between language and extralinguistic cognition using a naturalistic fMRI paradigm. In our experiment, 28 participants listened to a specially crafted dialogical narrative with a controlled number of spatial demonstratives. A fast multiband-EPI acquisition sequence (TR = 388 m s) combined with finite impulse response (FIR) modelling of the hemodynamic response was used to capture signal changes at word-level resolution. We found that spatial demonstratives bilaterally engage a network of parietal areas, including the supramarginal gyrus, the angular gyrus, and precuneus, implicated in information integration and visuospatial processing. Moreover, demonstratives recruit frontal regions, including the right FEF, implicated in attentional orienting and reference frames shifts. Finally, using multivariate similarity analyses, we provide evidence for a general involvement of the dorsal (“where”) stream in the processing of spatial expressions, as opposed to ventral pathways encoding object semantics. Overall, our results suggest that language processing relies on a distributed architecture, recruiting neural resources for perception, attention, and extra-linguistic aspects of cognition in a dynamic and context-dependent fashion