Appendix A: “The Basic Postulates of Accounting”

Functional Magnetic Resonance Imaging (fMRI) is a relatively new imaging technique, first reported in 1992, which enables mapping of brain functions with high spatial resolution. Functionally active areas are distinguished by a small signal increase mediated by changes in local blood oxygenation in response to neural activity. The ability to non-invasively map brain function and the large number of MRI scanners quickly made the method very popular, and fMRI have had a huge impact on the study of brain function, both in healthy and diseased subjects. The most common clinical application of fMRI is pre-surgical mapping of brain functions in order to optimise surgical interventions. The clinical fMRI examination procedure can be divided into four integrated parts: (1) patient preparation, (2) image acquisition, (3) image analysis and (4) clinical decision. In this thesis, important aspects of all parts of the fMRI examination procedure are explored with the aim to provide recommendations and methods for prosperous clinical usage of the technique. The most important results of the thesis were: (I) administration of low doses of diazepam to reduce anxiety did not invalidate fMRI mapping results of primary motor and language areas, (II) the choice of visual stimuli equipment can have severe impact on the mapping of visual areas, (III) three-dimensional fMRI imaging sequences did not perform better than two-dimensional imaging sequences, (IV) adaptive spatial filtering can improve the fMRI data analysis, (V) clinical decisions should not be based on activation results from a single statistical threshold

A spatiotemporal nonparametric Bayesian model of multi-subject fMRI data

In this paper we propose a unified, probabilistically coherent framework for the analysis of task-related brain activity in multi-subject fMRI experiments. This is distinct from two-stage “group analysis” approaches traditionally considered in the fMRI literature, which separate the inference on the individual fMRI time courses from the inference at the population level. In our modeling approach we consider a spatiotemporal linear regression model and specifically account for the between-subjects heterogeneity in neuronal activity via a spatially informed multi-subject nonparametric variable selection prior. For posterior inference, in addition to Markov chain Monte Carlo sampling algorithms, we develop suitable variational Bayes algorithms. We show on simulated data that variational Bayes inference achieves satisfactory results at more reduced computational costs than using MCMC, allowing scalability of our methods. In an application to data collected to assess brain responses to emotional stimuli our method correctly detects activation in visual areas when visual stimuli are presented

Representational Similarity Analysis – Connecting the Branches of Systems Neuroscience

A fundamental challenge for systems neuroscience is to quantitatively relate its three major branches of research: brain-activity measurement, behavioral measurement, and computational modeling. Using measured brain-activity patterns to evaluate computational network models is complicated by the need to define the correspondency between the units of the model and the channels of the brain-activity data, e.g., single-cell recordings or voxels from functional magnetic resonance imaging (fMRI). Similar correspondency problems complicate relating activity patterns between different modalities of brain-activity measurement (e.g., fMRI and invasive or scalp electrophysiology), and between subjects and species. In order to bridge these divides, we suggest abstracting from the activity patterns themselves and computing representational dissimilarity matrices (RDMs), which characterize the information carried by a given representation in a brain or model. Building on a rich psychological and mathematical literature on similarity analysis, we propose a new experimental and data-analytical framework called representational similarity analysis (RSA), in which multi-channel measures of neural activity are quantitatively related to each other and to computational theory and behavior by comparing RDMs. We demonstrate RSA by relating representations of visual objects as measured with fMRI in early visual cortex and the fusiform face area to computational models spanning a wide range of complexities. The RDMs are simultaneously related via second-level application of multidimensional scaling and tested using randomization and bootstrap techniques. We discuss the broad potential of RSA, including novel approaches to experimental design, and argue that these ideas, which have deep roots in psychology and neuroscience, will allow the integrated quantitative analysis of data from all three branches, thus contributing to a more unified systems neuroscience

Visual Field Maps of the Visual Cortex

fMRI imaging measures brain activity by detecting changes in the brain that are associated with blood flow and can be used in order to determine the size and location of visual field maps. This study measured BOLD responses in humans to understand spatial summation by showing them spatial contrast pattern images through vertical and horizontal apertures.Nonlinear responses can be computed from the visual field responses, from which the spatial summation ratio is calculated. A ratio of less than one indicates a smaller response to exposure to a full aperture or contrast image than predicted by the linear model. Datasets were analyzed using MATLAB, producing eccentricity maps of the visual cortices from stimuli. BOLD response curves identify the intensity of the maps within a particular voxel. The eccentricity maps can then be overlain on anatomical representations of the brain, characterizing the various visual field cortices. Methods of fMRI data analysis are confirmed with the results of this study

Aberrant activity in conceptual networks underlies N400 deficits and unusual thoughts in schizophrenia.

BackgroundThe N400 event-related potential (ERP) is triggered by meaningful stimuli that are incongruous, or unmatched, with their semantic context. Functional magnetic resonance imaging (fMRI) studies have identified brain regions activated by semantic incongruity, but their precise links to the N400 ERP are unclear. In schizophrenia (SZ), N400 amplitude reduction is thought to reflect overly broad associations in semantic networks, but the abnormalities in brain networks underlying deficient N400 remain unknown. We utilized joint independent component analysis (JICA) to link temporal patterns in ERPs to neuroanatomical patterns from fMRI and investigate relationships between N400 amplitude and neuroanatomical activation in SZ patients and healthy controls (HC).MethodsSZ patients (n = 24) and HC participants (n = 25) performed a picture-word matching task, in which words were either matched (APPLE→apple) by preceding pictures, or were unmatched by semantically related (in-category; IC, APPLE→lemon) or unrelated (out of category; OC, APPLE→cow) pictures, in separate ERP and fMRI sessions. A JICA "data fusion" analysis was conducted to identify the fMRI brain regions specifically associated with the ERP N400 component. SZ and HC loading weights were compared and correlations with clinical symptoms were assessed.ResultsJICA identified an ERP-fMRI "fused" component that captured the N400, with loading weights that were reduced in SZ. The JICA map for the IC condition showed peaks of activation in the cingulate, precuneus, bilateral temporal poles and cerebellum, whereas the JICA map from the OC condition was linked primarily to visual cortical activation and the left temporal pole. Among SZ patients, fMRI activity from the IC condition was inversely correlated with unusual thought content.ConclusionsThe neural networks associated with the N400 ERP response to semantic violations depends on conceptual relatedness. These findings are consistent with a distributed network underlying neural responses to semantic incongruity including unimodal visual areas as well as integrative, transmodal areas. Unusual thoughts in SZ may reflect impaired processing in transmodal hub regions such as the precuneus, leading to overly broad semantic associations

Reading visually embodied meaning from the brain: Visually grounded computational models decode visual-object mental imagery induced by written text

Embodiment theory predicts that mental imagery of object words recruits neural circuits involved in object perception. The degree of visual imagery present in routine thought and how it is encoded in the brain is largely unknown. We test whether fMRI activity patterns elicited by participants reading objects' names include embodied visual-object representations, and whether we can decode the representations using novel computational image-based semantic models. We first apply the image models in conjunction with text-based semantic models to test predictions of visual-specificity of semantic representations in different brain regions. Representational similarity analysis confirms that fMRI structure within ventral-temporal and lateral-occipital regions correlates most strongly with the image models and conversely text models correlate better with posterior-parietal/lateral-temporal/inferior-frontal regions. We use an unsupervised decoding algorithm that exploits commonalities in representational similarity structure found within both image model and brain data sets to classify embodied visual representations with high accuracy (8/10) and then extend it to exploit model combinations to robustly decode different brain regions in parallel. By capturing latent visual-semantic structure our models provide a route into analyzing neural representations derived from past perceptual experience rather than stimulus-driven brain activity. Our results also verify the benefit of combining multimodal data to model human-like semantic representations

Inverse transformed encoding models - A solution to the problem of correlated trial-by-trial parameter estimates in fMRI decoding

Techniques of multivariate pattern analysis (MVPA) can be used to decode the discrete experimental condition or a continuous modulator variable from measured brain activity during a particular trial. In functional magnetic resonance imaging (fMRI), trial-wise response amplitudes are sometimes estimated from the measured signal using a general linear model (GLM) with one onset regressor for each trial. When using rapid event-related designs with trials closely spaced in time, those estimates are highly variable and serially correlated due to the temporally extended shape of the hemodynamic response function (HRF). Here, we describe inverse transformed encoding modelling (ITEM), a principled approach of accounting for those serial correlations and decoding from the resulting estimates, at low computational cost and with no loss in statistical power. We use simulated data to show that ITEM outperforms the current standard approach in terms of decoding accuracy and analyze empirical data to demonstrate that ITEM is capable of visual reconstruction from fMRI signals

An Uncertainty Visual Analytics Framework for Functional Magnetic Resonance Imaging

Improving understanding of the human brain is one of the leading pursuits of modern scientific research. Functional magnetic resonance imaging (fMRI) is a foundational technique for advanced analysis and exploration of the human brain. The modality scans the brain in a series of temporal frames which provide an indication of the brain activity either at rest or during a task. The images can be used to study the workings of the brain, leading to the development of an understanding of healthy brain function, as well as characterising diseases such as schizophrenia and bipolar disorder. Extracting meaning from fMRI relies on an analysis pipeline which can be broadly categorised into three phases: (i) data acquisition and image processing; (ii) image analysis; and (iii) visualisation and human interpretation. The modality and analysis pipeline, however, are hampered by a range of uncertainties which can greatly impact the study of the brain function. Each phase contains a set of required and optional steps, containing inherent limitations and complex parameter selection. These aspects lead to the uncertainty that impacts the outcome of studies. Moreover, the uncertainties that arise early in the pipeline, are compounded by decisions and limitations further along in the process. While a large amount of research has been undertaken to examine the limitations and variable parameter selection, statistical approaches designed to address the uncertainty have not managed to mitigate the issues. Visual analytics, meanwhile, is a research domain which seeks to combine advanced visual interfaces with specialised interaction and automated statistical processing designed to exploit human expertise and understanding. Uncertainty visual analytics (UVA) tools, which aim to minimise and mitigate uncertainties, have been proposed for a variety of data, including astronomical, financial, weather and crime. Importantly, UVA approaches have also seen success in medical imaging and analysis. However, there are many challenges surrounding the application of UVA to each research domain. Principally, these involve understanding what the uncertainties are and the possible effects so they may be connected to visualisation and interaction approaches. With fMRI, the breadth of uncertainty arising in multiple stages along the pipeline and the compound effects, make it challenging to propose UVAs which meaningfully integrate into pipeline. In this thesis, we seek to address this challenge by proposing a unified UVA framework for fMRI. To do so, we first examine the state-of-the-art landscape of fMRI uncertainties, including the compound effects, and explore how they are currently addressed. This forms the basis of a field we term fMRI-UVA. We then present our overall framework, which is designed to meet the requirements of fMRI visual analysis, while also providing an indication and understanding of the effects of uncertainties on the data. Our framework consists of components designed for the spatial, temporal and processed imaging data. Alongside the framework, we propose two visual extensions which can be used as standalone UVA applications or be integrated into the framework. Finally, we describe a conceptual algorithmic approach which incorporates more data into an existing measure used in the fMRI analysis pipeline

Distinct functional activity of the precuneus and posterior cingulate cortex during encoding in the preclinical stage of Alzheimer's Disease

In this study functional magnetic resonance imaging (fMRI) is used to investigate the functional brain activation pattern in the preclinical stage of AD (pre-AD) subjects during a visual encoding memory task. Thirty subjects, eleven in the pre-AD stage, with decreased cerebrospinal fluid levels of Aβ42 (<500 pg/ml), and 19 controls with normal Aβ42 levels (CTR) were included. fMRI was acquired during a visual encoding task. Data were analyzed through an Independent Component Analysis (ICA) and region-of-interest-based univariate analysis of task-related BOLD signal change. From the ICA decomposition, we identified the main task-related component, which included the activation of visual associative areas and prefrontal executive regions, and the deactivation of the default-mode network. The activation was positively correlated with task performance in the CTR group (p < 0.0054). Within this pattern, subjects in the pre-AD stage had significantly greater activation of the precuneus and posterior cingulate cortex during encoding. Subjects in the pre-AD stage present distinct functional neural activity before the appearance of clinical symptomatology. These findings may represent that subtle changes in functional brain activity precede clinical and cognitive symptoms in the AD continuum. Present findings provide evidence suggesting that fMRI may be a suitable biomarker of preclinical AD

Brain state dynamics differ between eyes open and eyes closed rest

The human brain exhibits spatio-temporally complex activity even in the absence of external stimuli, cycling through recurring patterns of activity known as brain states. Thus far, brain state analysis has primarily been restricted to unimodal neuroimaging data sets, resulting in a limited definition of state and a poor understanding of the spatial and temporal relationships between states identified from different modalities. Here, we applied hidden Markov model (HMM) to concurrent electroencephalography-functional magnetic resonance imaging (EEG-fMRI) eyes open (EO) and eyes closed (EC) resting-state data, training models on the EEG and fMRI data separately, and evaluated the models' ability to distinguish dynamics between the two rest conditions. Additionally, we employed a general linear model approach to identify the BOLD correlates of the EEG-defined states to investigate whether the fMRI data could be used to improve the spatial definition of the EEG states. Finally, we performed a sliding window-based analysis on the state time courses to identify slower changes in the temporal dynamics, and then correlated these time courses across modalities. We found that both models could identify expected changes during EC rest compared to EO rest, with the fMRI model identifying changes in the activity and functional connectivity of visual and attention resting-state networks, while the EEG model correctly identified the canonical increase in alpha upon eye closure. In addition, by using the fMRI data, it was possible to infer the spatial properties of the EEG states, resulting in BOLD correlation maps resembling canonical alpha-BOLD correlations. Finally, the sliding window analysis revealed unique fractional occupancy dynamics for states from both models, with a selection of states showing strong temporal correlations across modalities. Overall, this study highlights the efficacy of using HMMs for brain state analysis, confirms that multimodal data can be used to provide more in-depth definitions of state and demonstrates that states defined across different modalities show similar temporal dynamics.</p