Consistency and similarity of MEG- and fMRI-signal time courses during movie viewing

Movie viewing allows human perception and cognition to be studied in complex, real-life-like situations in a brain-imaging laboratory. Previous studies with functional magnetic resonance imaging (fMRI) and with magneto-and electroencephalography (MEG and EEG) have demonstrated consistent temporal dynamics of brain activity across movie viewers. However, little is known about the similarities and differences of fMRI and MEG or EEG dynamics during such naturalistic situations. We thus compared MEG and fMRI responses to the same 15-min black-and-white movie in the same eight subjects who watched the movie twice during both MEG and fMRI recordings. We analyzed intra-and intersubject voxel-wise correlations within each imaging modality as well as the correlation of the MEG envelopes and fMRI signals. The fMRI signals showed voxel-wise within-and between-subjects correlations up to r = 0.66 and r = 0.37, respectively, whereas these correlations were clearly weaker for the envelopes of band-pass filtered (7 frequency bands below 100 Hz) MEG signals (within-subjects correlation r <0.14 and between-subjects r <0.05). Direct MEG-fMRI voxel-wise correlations were unreliable. Notably, applying a spatial-filtering approach to the MEG data uncovered consistent canonical variates that showed considerably stronger (up to r = 0.25) between-subjects correlations than the univariate voxel-wise analysis. Furthermore, the envelopes of the time courses of these variates up to about 10 Hz showed association with fMRI signals in a general linear model. Similarities between envelopes of MEG canonical variates and fMRI voxel time-courses were seen mostly in occipital, but also in temporal and frontal brain regions, whereas intra-and intersubject correlations for MEG and fMRI separately were strongest only in the occipital areas. In contrast to the conventional univariate analysis, the spatial-filtering approach was able to uncover associations between the MEG envelopes and fMRI time courses, shedding light on the similarities of hemodynamic and electromagnetic brain activities during movie viewing.Peer reviewe

Data-driven approaches in the investigation of social perception

The complexity of social perception poses a challenge to traditional approaches to understand its psychological and neurobiological underpinnings. Data-driven methods are particularly well suited to tackling the often high-dimensional nature of stimulus spaces and of neural representations that characterize social perception. Such methods are more exploratory, capitalize on rich and large datasets, and attempt to discover patterns often without strict hypothesis testing. We present four case studies here: behavioural studies on face judgements, two neuroimaging studies of movies, and eyetracking studies in autism. We conclude with suggestions for particular topics that seem ripe for data-driven approaches, as well as caveats and limitations

Altered EEG Oscillatory Brain Networks During Music-Listening in Major Depression

To examine the electrophysiological underpinnings of the functional networks involved in music listening, previous approaches based on spatial independent component analysis (ICA) have recently been used to ongoing electroencephalography (EEG) and magnetoencephalography (MEG). However, those studies focused on healthy subjects, and failed to examine the group-level comparisons during music listening. Here, we combined group-level spatial Fourier ICA with acoustic feature extraction, to enable group comparisons in frequency-specific brain networks of musical feature processing. It was then applied to healthy subjects and subjects with major depressive disorder (MDD). The music-induced oscillatory brain patterns were determined by permutation correlation analysis between individual time courses of Fourier-ICA components and musical features. We found that (1) three components, including a beta sensorimotor network, a beta auditory network and an alpha medial visual network, were involved in music processing among most healthy subjects; and that (2) one alpha lateral component located in the left angular gyrus was engaged in music perception in most individuals with MDD. The proposed method allowed the statistical group comparison, and we found that: (1) the alpha lateral component was activated more strongly in healthy subjects than in the MDD individuals, and that (2) the derived frequency-dependent networks of musical feature processing seemed to be altered in MDD participants compared to healthy subjects. The proposed pipeline appears to be valuable for studying disrupted brain oscillations in psychiatric disorders during naturalistic paradigms.Peer reviewe

Neuromagnetic activation and oscillatory dynamics of stimulus-locked processing during naturalistic viewing

Naturalistic stimuli such as watching a movie while in the scanner provide an ecologically valid paradigm that has the potential of extracting valuable information on how the brain processes complex stimuli in realistic visual and auditory contexts. Naturalistic viewing is also easier to conduct with challenging participant groups including patients and children. Given the high temporal resolution of MEG, in the present study, we demonstrate how a short movie clip can be used to map distinguishable activation and connectivity dynamics underlying the processing of specific classes of visual stimuli such as face and hand manipulations, as well as contrasting activation dynamics for auditory words and non-words. MEG data were collected from 22 healthy volunteers (6 females, 3 left handed, mean age – 27.7 ± 5.28 years) during the presentation of naturalistic audiovisual stimuli. The MEG data were split into trials with the onset of the stimuli belonging to classes of interest (words, non-words, faces, hand manipulations). Based on the components of the averaged sensor ERFs time-locked to the visual and auditory stimulus onset, four and three time-windows, respectively, were defined to explore brain activation dynamics. Pseudo-Z, defined as the ratio of the source-projected time-locked power to the projected noise power for each vertex, was computed and used as a proxy of time-locked brain activation. Statistical testing using the mean-centered Partial Least Squares analysis indicated periods where a given visual or auditory stimuli had higher activation. Based on peak pseudo-Z differences between the visual conditions, time-frequency resolved analyses were performed to assess beta band desynchronization in motor-related areas, and inter-trial phase synchronization between face processing areas. Our results provide the first evidence that activation and connectivity dynamics in canonical brain regions associated with the processing of particular classes of visual and auditory stimuli can be reliably mapped using MEG during presentation of naturalistic stimuli. Given the strength of MEG for brain mapping in temporal and frequency domains, the use of naturalistic stimuli may open new techniques in analyzing brain dynamics during ecologically valid sensation and perception

EEG in the classroom: Synchronised neural recordings during video presentation

We performed simultaneous recordings of electroencephalography (EEG) from multiple students in a classroom, and measured the inter-subject correlation (ISC) of activity evoked by a common video stimulus. The neural reliability, as quantified by ISC, has been linked to engagement and attentional modulation in earlier studies that used high-grade equipment in laboratory settings. Here we reproduce many of the results from these studies using portable low-cost equipment, focusing on the robustness of using ISC for subjects experiencing naturalistic stimuli. The present data shows that stimulus-evoked neural responses, known to be modulated by attention, can be tracked in for groups of students with synchronized EEG acquisition. This is a step towards real-time inference of engagement in the classroom.Comment: 14 pages, 5 figures, 3 tables. Preprint version. Revision of original preprint. Supplementary materials added as ancillary fil

Exploring Frequency-Dependent Brain Networks from Ongoing EEG Using Spatial ICA During Music Listening

Recently, exploring brain activity based on functional networks during naturalistic stimuli especially music and video represents an attractive challenge because of the low signal-to-noise ratio in collected brain data. Although most efforts focusing on exploring the listening brain have been made through functional magnetic resonance imaging (fMRI), sensor-level electro- or magnetoencephalography (EEG/MEG) technique, little is known about how neural rhythms are involved in the brain network activity under naturalistic stimuli. This study exploited cortical oscillations through analysis of ongoing EEG and musical feature during freely listening to music. We used a data-driven method that combined music information retrieval with spatial Fourier Independent Components Analysis (spatial Fourier-ICA) to probe the interplay between the spatial profiles and the spectral patterns of the brain network emerging from music listening. Correlation analysis was performed between time courses of brain networks extracted from EEG data and musical feature time series extracted from music stimuli to derive the musical feature related oscillatory patterns in the listening brain. We found brain networks of musical feature processing were frequency-dependent. Musical feature time series, especially fluctuation centroid and key feature, were associated with an increased beta activation in the bilateral superior temporal gyrus. An increased alpha oscillation in the bilateral occipital cortex emerged during music listening, which was consistent with alpha functional suppression hypothesis in task-irrelevant regions. We also observed an increased delta-beta oscillatory activity in the prefrontal cortex associated with musical feature processing. In addition to these findings, the proposed method seems valuable for characterizing the large-scale frequency-dependent brain activity engaged in musical feature processing.Peer reviewe

Decoding the consumer’s brain: Neural representations of consumer experience

Understanding consumer experience – what consumers think about brands, how they feel about services, whether they like certain products – is crucial to marketing practitioners. ‘Neuromarketing’, as the application of neuroscience in marketing research is called, has generated excitement with the promise of understanding consumers’ minds by probing their brains directly. Recent advances in neuroimaging analysis leverage machine learning and pattern classification techniques to uncover patterns from neuroimaging data that can be associated with thoughts and feelings. In this dissertation, I measure brain responses of consumers by functional magnetic resonance imaging (fMRI) in order to ‘decode’ their mind. In three different studies, I have demonstrated how different aspects of consumer experience can be studied with fMRI recordings. First, I study how consumers think about brand image by comparing their brain responses during passive viewing of visual templates (photos depicting various social scenarios) to those during active visualizing of a brand’s image. Second, I use brain responses during viewing of affective pictures to decode emotional responses during watching of movie-trailers. Lastly, I examine whether marketing videos that evoke s

Decreased intersubject synchrony in dynamic valence ratings of sad movie contents in dysphoric individuals

Emotional reactions to movies are typically similar between people. However, depressive symptoms decrease synchrony in brain responses. Less is known about the effect of depressive symptoms on intersubject synchrony in conscious stimulus-related processing. In this study, we presented amusing, sad and fearful movie clips to dysphoric individuals (those with elevated depressive symptoms) and control participants to dynamically rate the clips' valences (positive vs. negative). We analysed both the valence ratings' mean values and intersubject correlation (ISC). We used electrodermal activity (EDA) to complement the measurement in a separate session. There were no group differences in either the EDA or mean valence rating values for each movie type. As expected, the valence ratings' ISC was lower in the dysphoric than the control group, specifically for the sad movie clips. In addition, there was a negative relationship between the valence ratings' ISC and depressive symptoms for sad movie clips in the full sample. The results are discussed in the context of the negative attentional bias in depression. The findings extend previous brain activity results of ISC by showing that depressive symptoms also increase variance in conscious ratings of valence of stimuli in a mood-congruent manner.Peer reviewe

Building Mental Experiences: From Scenes to Events

Mental events are central to everyday cognition, be it our continuous perception of the world, recalling autobiographical memories, or imagining the future. Little is known about the fine-grained temporal dynamics of these processes. Given the apparent predominance of scene imagery across cognition, in this thesis I used magnetoencephalography to investigate whether and how activity in the hippocampus and ventromedial prefrontal cortex (vmPFC) supports the mental construction of scenes and the events to which they give rise. In the first experiment, participants gradually imagined scenes and also closely matched non-scene arrays; this allowed me to assess whether any brain regions showed preferential responses to scene imagery. The anterior hippocampus and vmPFC were particularly engaged by the construction of scene imagery, with the vmPFC driving hippocampal activity. In the second experiment, I found that certain objects – those that were space-defining – preferentially engaged the vmPFC and superior temporal gyrus during scene construction, providing insight into how objects affect the creation of scene representations. The third experiment involved boundary extension during scene perception, permitting me to examine how single scenes might be prepared for inclusion into events. I observed changes in evoked responses just 12.5-58 ms after scene onset over fronto-temporal sensors, with again the vmPFC exerting a driving influence on other brain regions, including the hippocampus. In the final experiment, participants watched brief movies of events built from a series of scenes or non-scene patterns. A difference in evoked responses between the two event types emerged during the first frame of the movies, the primary source of which was shown to be the hippocampus. The enduring theme of the results across experiments was scene-specific engagement of the hippocampus and vmPFC, with the latter being the driving influence. Overall, this thesis provides insights into the neural dynamics of how scenes are built, made ready for inclusion into unfolding mental episodes, and then linked to produce our seamless experience of the world