49 research outputs found
CONTEXT EFFECTS ON THE NEURAL CORRELATES OF EMOTION
Human emotions are inextricably linked to the context in which they occur, yet neuroscience research on emotion often overlooks the role of context in shaping the neural correlates of human emotions. This dissertation, through three studies, begins to address this gap. Study 1 (Chapter 2) investigated how language as a form of context influences the brainâs response to facial expressions of anger and disgust, showing distinct differences in neural activity related to disgust perception between Chinese Asian and White American participants. Language was found to play a significant role in shaping the neural correlates of emotion perception, particularly in the context of disgust for Chinese Asian participants. Study 2 (Chapter 3) examined the influence of social and cultural contexts on the neural correlates of fear and sadness, demonstrating that both cultural group membership and cultural attitudes are related to the brainâs processing of negative emotional experiences. Study 3 (Chapter 4) focused on individual and group level variation in emotion-related functional connectivity, finding both idiographic and nomothetic patterns of connectivity related to negative emotional experiences. It further highlighted the influence of cultural attitudes on the neural correlates of emotion. Overall, these studies illustrate the importance of considering diverse contextual variables in studying the neural basis of emotion. They show that social and cultural contexts influence how the human brain processes and represents emotions.Doctor of Philosoph
Mismatch responses: Probing probabilistic inference in the brain
Sensory signals are governed by statistical regularities and carry valuable information about the unfolding of environmental events. The brain is thought to capitalize on the probabilistic nature of sequential inputs to infer on the underlying (hidden) dynamics driving sensory stimulation. Mis-match responses (MMRs) such as the mismatch negativity (MMN) and the P3 constitute prominent neuronal signatures which are increasingly interpreted as reflecting a mismatch between the current sensory input and the brainâs generative model of incoming stimuli. As such, MMRs might be viewed as signatures of probabilistic inference in the brain and their response dynamics can provide insights into the underlying computational principles. However, given the dominance of the auditory modality in MMR research, the specifics of brain responses to probabilistic sequences across sensory modalities and especially in the somatosensory domain are not well characterized.
The work presented here investigates MMRs across the auditory, visual and somatosensory modality by means of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). We designed probabilistic stimulus sequences to elicit and characterize MMRs and employed computational modeling of response dynamics to inspect different aspects of the brainâs generative model of the sensory environment. In the first study, we used a volatile roving stimulus paradigm to elicit somatosensory MMRs and performed single-trial modeling of EEG signals in sensor and source space. Model comparison suggested that responses reflect Bayesian inference based on the estimation of transition probability and limited information integration of the recent past in order to adapt to a changing environment. The results indicated that somatosensory MMRs reflect an initial mismatch between sensory input and model beliefs represented by confidence-corrected surprise (CS) followed by model adjustment dynamics represented by Bayesian surprise (BS). For the second and third study we designed a tri-modal roving stimulus paradigm to delineate modality specific and modality general features of mismatch processing. Computational modeling of EEG signals in study 2 suggested that single-trial dynamics reflect Bayesian inference based on estimation of uni-modal transition probabilities as well as cross-modal conditional dependencies. While early mismatch processing around the MMN tended to reflect CS, later MMRs around the P3 rather reflect BS, in correspondence to the somatosensory study. Finally, the fMRI results of study 3 showed that MMRs are generated by an interaction of modality specific regions in higher order sensory cortices and a modality general fronto-parietal network. Inferior parietal regions in particular were sensitive to expectation violations with respect to the cross-modal contingencies in the stimulus sequences. Overall, our results indicate that MMRs across the senses reflect processes of probabilistic inference in a complex and inherently multi-modal environment.Sensorische Signale sind durch statistische RegularitĂ€ten bestimmt und beinhalten wertvolle Informationen ĂŒber die Entwicklung von Umweltereignissen. Es wird angenommen, dass das Gehirn die Wahrscheinlichkeitseigenschaften sequenzieller Reize nutzt um auf die zugrundeliegenden (verborgenen) Dynamiken zu schlieĂen, welche sensorische Stimulation verursachen. Diskrepanz-Reaktionen ("Mismatch responses"; MMRs) wie die "mismatch negativity" (MMN) und die P3 sind bekannte neuronale Signaturen die vermehrt als Signale einer Diskrepanz zwischen der momentanen sensorischen Einspeisung und dem generativen Modell, welches das Gehirn von den eingehenden Reizen erstellt angesehen werden. Als solche können MMRs als Signaturen von wahrscheinlichkeitsbasierter Inferenz im Gehirn betrachtet werden und ihre Reaktionsdynamiken können Einblicke in die zugrundeliegenden komputationalen Prinzipien geben. Angesichts der Dominanz der auditorischen ModalitĂ€t in der MMR-Forschung, sind allerdings die spezifischen Eigenschaften von Hirn-Reaktionen auf Wahrscheinlichkeitssequenzen ĂŒber sensorische ModalitĂ€ten hinweg und vor allem in der somatosensorischen ModalitĂ€t nicht gut charakterisiert.
Die hier vorgestellte Arbeit untersucht MMRs ĂŒber die auditorische, visuelle und somatosensorische ModalitĂ€t hinweg anhand von Elektroenzephalographie (EEG) und funktioneller Magnetresonanztomographie (fMRT). Wir gestalteten wahrscheinlichkeitsbasierte Reizsequenzen, um MMRs auszulösen und zu charakterisieren und verwendeten komputationale Modellierung der Reaktionsdynamiken, um verschiedene Aspekte des generativen Modells des Gehirns von der sensorischen Umwelt zu untersuchen. In der ersten Studie verwendeten wir ein volatiles "Roving-Stimulus"-Paradigma, um somatosensorische MMRs auszulösen und modellierten die Einzel-Proben der EEG-Signale im sensorischen und Quell-Raum. Modellvergleiche legten nahe, dass die Reaktionen Bayesâsche Inferenz abbilden, basierend auf der SchĂ€tzung von Transitionswahrscheinlichkeiten und limitierter Integration von Information der jĂŒngsten Vergangenheit, welche eine Anpassung an UmweltĂ€nderungen ermöglicht. Die Ergebnisse legen nahe, dass somatosen-sorische MMRs eine initiale Diskrepanz zwischen sensorischer Einspeisung und ModellĂŒberzeugung reflektieren welche durch "confidence-corrected surprise" (CS) reprĂ€sentiert ist, gefolgt von Modelanpassungsdynamiken reprĂ€sentiert von "Bayesian surprise" (BS). FĂŒr die zweite und dritte Studie haben wir ein Tri-Modales "Roving-Stimulus"-Paradigma gestaltet, um modalitĂ€tsspezifische und modalitĂ€tsĂŒbergreifende Eigenschaften von Diskrepanzprozessierung zu umreiĂen. Komputationale Modellierung von EEG-Signalen in Studie 2 legte nahe, dass Einzel-Proben Dynamiken Bayesâsche Inferenz abbilden, basierend auf der SchĂ€tzung von unimodalen Transitionswahrscheinlichkeiten sowie modalitĂ€tsĂŒbergreifenden bedingten AbhĂ€ngigkeiten. WĂ€hrend frĂŒhe Diskrepanzprozessierung um die MMN dazu tendierten CS zu reflektieren, so reflektierten spĂ€tere MMRs um die P3 eher BS, in Ăbereinstimmung mit der somatosensorischen Studie. AbschlieĂend zeigten die fMRT-Ergebnisse der Studie 3 dass MMRs durch eine Interaktion von modalitĂ€tsspezifischen Regionen in sensorischen Kortizes höherer Ordnung mit einem modalitĂ€tsĂŒbergreifenden fronto-parietalen Netzwerk generiert werden. Inferior parietale Regionen im Speziellen waren sensitiv gegenĂŒber ErwartungsverstoĂ in Bezug auf die modalitĂ€tsĂŒbergreifenden Wahrscheinlichkeiten in den Reizsequenzen. Insgesamt weisen unsere Ergebnisse darauf hin, dass MMRs ĂŒber die Sinne hinweg Prozesse von wahrscheinlichkeitsbasierter Inferenz in einer komplexen und inhĂ€rent multi-modalen Umwelt darstellen
Learning Identifiable Representations: Independent Influences and Multiple Views
Intelligent systems, whether biological or artificial, perceive unstructured information from the world around them: deep neural networks designed for object recognition receive collections of pixels as inputs; living beings capture visual stimuli through photoreceptors that convert incoming light into electrical signals. Sophisticated signal processing is required to extract meaningful features (e.g., the position, dimension, and colour of objects in an image) from these inputs: this motivates the field of representation learning. But what features should be deemed meaningful, and how to learn them?
We will approach these questions based on two metaphors. The first one is the cocktail-party problem, where a number of conversations happen in parallel in a room, and the task is to recover (or separate) the voices of the individual speakers from recorded mixturesâalso termed blind source separation. The second one is what we call the independent-listeners problem: given two listeners in front of some loudspeakers, the question is whether, when processing what they hear, they will make the same information explicit, identifying similar constitutive elements. The notion of identifiability is crucial when studying these problems, as it specifies suitable technical assumptions under which representations are uniquely determined, up to tolerable ambiguities like latent source reordering. A key result of this theory is that, when the mixing is nonlinear, the model is provably non-identifiable. A first question is, therefore, under what additional assumptions (ideally as mild as possible) the problem becomes identifiable; a second one is, what algorithms can be used to estimate the model.
The contributions presented in this thesis address these questions and revolve around two main principles. The first principle is to learn representation where the latent components influence the observations independently. Here the term âindependentlyâ is used in a non-statistical senseâwhich can be loosely thought of as absence of fine-tuning between distinct elements of a generative process. The second principle is that representations can be learned from paired observations or views, where mixtures of the same latent variables are observed, and they (or a subset thereof) are perturbed in one of the viewsâalso termed multi-view setting. I will present work characterizing these two problem settings, studying their identifiability and proposing suitable estimation algorithms. Moreover, I will discuss how the success of popular representation learning methods may be explained in terms of the principles above and describe an application of the second principle to the statistical analysis of group studies in neuroimaging
A multimodal cortical network of sensory expectation violation revealed by fMRI
The brain is subjected to multi-modal sensory information in an environment governed by statistical dependencies. Mismatch responses (MMRs), classically recorded with EEG, have provided valuable insights into the brain's processing of regularities and the generation of corresponding sensory predictions. Only few studies allow for comparisons of MMRs across multiple modalities in a simultaneous sensory stream and their corresponding cross-modal context sensitivity remains unknown. Here, we used a tri-modal version of the roving stimulus paradigm in fMRI to elicit MMRs in the auditory, somatosensory and visual modality. Participants (Nâ=â29) were simultaneously presented with sequences of low and high intensity stimuli in each of the three senses while actively observing the tri-modal input stream and occasionally reporting the intensity of the previous stimulus in a prompted modality. The sequences were based on a probabilistic model, defining transition probabilities such that, for each modality, stimuli were more likely to repeat (pâ=â.825) than change (pâ=â.175) and stimulus intensities were equiprobable (pâ=â.5). Moreover, each transition was conditional on the configuration of the other two modalities comprising global (cross-modal) predictive properties of the sequences. We identified a shared mismatch network of modality general inferior frontal and temporo-parietal areas as well as sensory areas, where the connectivity (psychophysiological interaction) between these regions was modulated during mismatch processing. Further, we found deviant responses within the network to be modulated by local stimulus repetition, which suggests highly comparable processing of expectation violation across modalities. Moreover, hierarchically higher regions of the mismatch network in the temporo-parietal area around the intraparietal sulcus were identified to signal cross-modal expectation violation. With the consistency of MMRs across audition, somatosensation and vision, our study provides insights into a shared cortical network of uni- and multi-modal expectation violation in response to sequence regularities
Examining the Relationships Between Socio-cognitive Factors and Neural Synchrony During Movie Watching Across Development
While different cognitive abilities mature, the conscious experiences of children likely become richer and more elaborate. A challenge in investigating relationships between cognitive development and real-world experiences is having measures that assess naturalistic processing. Movie watching offers a solution, since following the plot of a film requires cognitive processes that are similar to real-world experiences. When different adults watch the same film, their brain activity begins to align (known as neural synchrony). The strength of this alignment has been shown to reflect the degree to which different individuals are having a similar experience of the movie. While this phenomenon has been established in adults, much less is known about the neural mechanisms supporting naturalistic processing in children and adolescents. The current thesis investigated the neural correlates of movie watching across late childhood and early adolescence. In Chapter 2, I found that autistic children showed more variable brain responses in regions associated with social cognition when watching a movie compared to children without autism. In Chapter 3, I found that adolescents (ages 11-15) with higher cognitive scores showed greater neural synchrony during movie watching in brain regions associated with social processing and executive functions compared to those with below average cognitive scores. This pattern was not evident in children (ages 7-11) who differed in their cognitive scores. In Chapter 4, I found that although the spatial topographies of childrenâs functional brain networks were nearly indistinguishable during movie watching and rest, these two states differed in the degree of neural synchrony that was present within much of the brain. That is, movies led to significantly more neural synchrony compared to rest, except for in parts of the prefrontal cortex. Taken together, these results suggest that 1) autistic children have more distinct experiences when processing naturalistic stimuli compared to those without autism, 2) adolescents with higher cognitive scores have more similar experiences with each other when watching a movie compared to those with lower scores, and 3) although childrenâs brain networks during movie watching and rest have a similar functional architecture, processing a film leads to neural synchrony, whereas resting state does not
While you were sleeping: Evidence for high-level executive processing of an auditory narrative during sleep
During sleep we lack conscious awareness of the external environment. Yet, our internal mental state suggests that high-level cognitive processes persist. The nature and extent to which the external environment is processed during sleep remain largely unexplored. Here, we used an fMRI synchronization-based approach to examine responses to a narrative during wakefulness and sleep. The stimulus elicited the auditory network and a frontoparietal pattern of activity, consistent with high-level narrative plot-following. During REM sleep, the same frontoparietal pattern was observed in one of three participants, and partially in one other, confirming that it is possible to track and follow the moment-to-moment complexities of a narrative during REM sleep. Auditory network recruitment was observed in both non-REM and REM sleep, demonstrating preservation of low-level auditory processing, even in deep sleep. This novel approach investigating cognitive processing at different levels of awareness demonstrates that the brain can meaningfully process the external environment during REM sleep
Interpretable Domain-Aware Learning for Neuroimage Classification
In this thesis, we propose three interpretable domain-aware machine learning approaches to
analyse large-scale neuroimaging data from multiple domains, e.g. multiple centres and/or demographic groups. We focus on two questions: how to learn general patterns across domains, and how to learn domain-specific patterns.
Our first approach develops a feature-classifier adaptation framework for semi-supervised domain adaptation on brain decoding tasks. Based on this empirical study, we derive a dependence-based generalisation bound to guide the design of domain-aware learning algorithms. This theoretical result leads to the next two approaches. The covariate-independence regularisation approach is for learning domain-generic patterns. Incorporating hinge and least squares loss generates two covariate-independence regularised classifiers, whose superiority are validated by the experimental results on brain decoding tasks for unsupervised multi-source domain adaptation. The covariate-dependent learning approach is for learning domain-specific patterns, which can learn gender-specific patterns of brain lateralisation via employing the logistic loss.
Interpretability is often essential for neuroimaging tasks. Therefore, all three domain-aware learning approaches are primarily designed to produce linear, interpretable models. These domain-aware learning approaches offer feasible ways to learn interpretable general or specific patterns from multi-domain neuroimaging data for neuroscientists to gain insights. With source code released on GitHub, this work will accelerate data-driven neuroimaging studies and advance multi-source domain adaptation research
Ubiquitous Technologies for Emotion Recognition
Emotions play a very important role in how we think and behave. As such, the emotions we feel every day can compel us to act and influence the decisions and plans we make about our lives. Being able to measure, analyze, and better comprehend how or why our emotions may change is thus of much relevance to understand human behavior and its consequences. Despite the great efforts made in the past in the study of human emotions, it is only now, with the advent of wearable, mobile, and ubiquitous technologies, that we can aim to sense and recognize emotions, continuously and in real time. This book brings together the latest experiences, findings, and developments regarding ubiquitous sensing, modeling, and the recognition of human emotions
The Phenomenology, Pathophysiology and Progression of the Core Features of Lewy Body Dementia
Lewy body dementias â Dementia with Lewy bodies (DLB) and Parkinsonâs disease dementia (PDD) - are disabling neurodegenerative conditions defined pathologically by the presence of intraneuronal α-synuclein rich aggregates (âLewy bodiesâ and âLewy neuritesâ). These disorders are characterized by a set of âcoreâ clinical features, namely cognitive fluctuations, visual hallucinations, motor parkinsonism, and most recently added, REM sleep behaviour disorder. These features are central to the diagnosis of Lewy bodies dementias (especially DLB) and discriminate them from other neurodegenerative disorders. Despite decades of research, the etiopathogenesis underlying Lewy body disorders is poorly understood. This accounts for the relative lack of objective biomarkers and both symptomatic and disease modifying therapies. The present thesis comprises a series of investigations that seeks to understand the phenomenology, pathophysiology, and clinical progression of Lewy body dementias through focus on each of the core clinical features. Systematic review and empiric studies are organized under the respective headings of cognitive fluctuations, visual hallucinations, REM sleep behaviour disorder, motor features, interrelationships, and clinical progression of the core features. Novel clinical and pathophysiological insights are obtained which have implications for the prediction and diagnosis of core features, the development of new objective biomarkers, and clinical endpoints of disease progression. From these studies, a shared pathophysiological basis for the core features is postulated and potential avenues for future directions are highlighted, focusing on replication and validation of new biomarkers and clinical measures, discovery of new biomarkers and mechanisms, and translation to prodromal and patient cohorts
The relationship between emotion and memory: exploring the effects of valence across the adult life-span
Emotional items are often remembered better than neutral items regardless of valence
however, positive and negative emotions sometimes lead to differential effects on memory.
This variance is particularly prominent between laboratory-based memories and real-life
autobiographical memories. Additionally, valence-related patterns on memory are known to
change with age; older adults frequently demonstrate a preference for positive over negative
information. According to the Socio-Emotional Selectivity Theory (SST), this positivity effect
is the consequence of emotion-related goals becoming more important as future time
perspective decreases with age. Although the theory provides an explanation for age-related
differences in emotional processing, not all of the SSTâs predictions have been supported.
This thesis therefore explores the relationship between emotion and memory across
the adult life-span. Specifically, Study 1 examines autobiographical memories for an
emotional event to understand whether valence leads to differences in memory and if they
differ as a function of chronological age. Study 2 explores this relationship further in a
controlled laboratory setting and obtains neural and behavioural measures to test the
predictions of the SST. Meanwhile, Study 3 expands on Study 2 to evaluate the predictions of
the SST across three separate measures of emotional processing: memory, neural activation
and emotional well-being.
Overall, the results from Study 1 showed differential effects of valence on memory
for autobiographical memories, but failed to find age-related effects to support the SST. However, in Study 2, older adults exhibited the positivity effect in memory which may be
explained by the significant age-related differences in neural activity during emotional
processing. Finally, age-related differences were found in emotional well-being (Study 3),
however, not all predictions of the SST were supported, particularly the concept of future
time perspective. In summary, mixed support for the SST was found suggesting the theory
may not fully account for all age-related differences in emotional processing