Modeling and analysis of mechanisms underlying high-resolution functional MRI of cortical columns

High spatial resolution functional MRI (fMRI) and advanced multivariate analysis techniques are promising tools for studying the cortical basis of human cognitive processes at the level of columns and layers. However the true spatial specificity of high-resolution fMRI has not been quantified, and the basis for decoding from fine scale structures using large voxels and relatively low magnetic field strength is unknown. It is also not yet known what method and voxel size is optimal for decoding and what voxel size is optimal for high-resolution imaging. In this thesis we present four studies that answer part of these questions using a model-based approach of imaging cortical columns. We started our investigation of model-based analysis of high-resolution fMRI of cortical columns by addressing the specific problem of how it is possible to decode information thought to be mediated by cortical columns using large voxels at low field strength. Multivariate machine learning algorithms applied to human functional MRI (fMRI) data can decode information conveyed by cortical columns, despite the voxel-size being large relative to the width of columns. Several mechanisms have been proposed to underlie decoding of stimulus orientation or the stimulated eye. These include: (I) aliasing of high spatial-frequency components, including the main frequency component of the columnar organization, (II) contributions from local irregularities in the columnar organization, (III) contributions from large-scale non-columnar organizations, (IV) functionally selective veins with biased draining regions, and (V) complex spatio-temporal filtering of neuronal activity by fMRI voxels. Here we sought to assess the plausibility of two of the suggested mechanisms: (I) aliasing and (II) local irregularities, using a naive model of BOLD as blurring and MRI voxel sampling. To this end, we formulated a mathematical model that encompasses both the processes of imaging ocular dominance (OD) columns and the subsequent linear classification analysis. Through numerical simulations of the model, we evaluated the distribution of functional differential contrasts that can be expected when considering the pattern of cortical columns, the hemodynamic point spread function, the voxel size, and the noise. We found that with data acquisition parameters used at 3 Tesla, sub-voxel supra-Nyquist frequencies, including frequencies near the main frequency of the OD organization (0.5 cycles per mm), cannot contribute to the differential contrast. The differential functional contrast of local origin is dominated by low-amplitude contributions from low frequencies, associated with irregularities of the cortical pattern. Realizations of the model with parameters that reflected a best-case scenario and the reported BOLD point-spread at 3 Tesla (3.5 mm) predicted decoding performances lower than those that have been previously obtained at this magnetic field strength. We conclude that low frequency components that underlie local irregularities in the columnar organization are likely to play a role in decoding. We further expect that fMRI-based decoding relies, in part, on signal contributions from large-scale, non-columnar functional organizations, and from complex spatio-temporal filtering of neuronal activity by fMRI voxels, involving biased venous responses. Our model can potentially be used for evaluating and optimizing data-acquisition parameters for decoding information conveyed by cortical columns. Having developed a model of imaging ODCs we then used this model to estimate the spatial specificity of BOLD fMRI, specifically at high field (7 T). Previous attempts at characterizing the spatial specificity of the blood oxygenation level dependent functional MRI (BOLD fMRI) response by estimating its point-spread function (PSF) have conventionally relied on spatial representations of visual stimuli in area V1. Consequently, their estimates were confounded by the width and scatter of receptive fields of V1 neurons. Here, we circumvent these limits by instead using the inherent cortical spatial organization of ocular dominance columns (ODCs) to determine the PSF for both Gradient Echo (GE) and Spin Echo (SE) BOLD imaging at 7 Tesla. By applying Markov Chain Monte Carlo sampling on a probabilistic generative model of imaging ODCs, we quantified the PSFs that best predict the spatial structure and magnitude of differential ODCs’ responses. Prior distributions for the ODC model parameters were determined by analyzing published data of cytochrome oxidase patterns from post-mortem histology of human V1 and of neurophysiological ocular dominance indices. The most probable PSF full-widths at half-maximum were 0.82 mm (SE) and 1.02 mm (GE). Our results provide a quantitative basis for the spatial specificity of BOLD fMRI at ultra-high fields, which can be used for planning and interpretation of high-resolution differential fMRI of fine-scale cortical organizations. Our BOLD fMRI PSF findings show that the PSF is considerably smaller than what was reported previously. This in turn raised the question of the role of the imaging PSF, which now has become relevant. Next, we show that the commonly used magnitude point-spread function fails to accurately represent the true effects of k-space sampling and signal decay, and propose an alternative model that accounts more accurately for these effects. The effects of k-space sampling and signal decay on the effective spatial resolution of MRI and functional MRI (fMRI) are commonly assessed by means of the magnitude point-spread function (PSF), defined as the absolute values (magnitudes) of the complex MR imaging PSF. It is commonly assumed that this magnitude PSF signifies blurring, which can be quantified by its full-width at half-maximum (FWHM). Here we show that the magnitude PSF fails to accurately represent the true effects of k-space sampling and signal decay. Firstly, a substantial part of the width of the magnitude PSF is due to MRI sampling per se. This part is independent of any signal decay and its effect depends on the spatial frequency composition of the imaged object. Therefore, it cannot always be expected to introduce blurring. Secondly, MRI reconstruction is typically followed by taking the absolute values (magnitude image) of the reconstructed complex image. This introduces a non-linear stage into the process of image formation. The complex imaging PSF does not fully describe this process, since it does not reflect the stage of taking the magnitude image. Its corresponding magnitude PSF fails to correctly describe this process, since convolving the original pattern with the magnitude PSF is different from the true process of taking the absolute following a convolution with the complex imaging PSF. Lastly, signal decay can have not only a blurring, but also a high-pass filtering effect. This cannot be reflected by the strictly positive width of the magnitude PSF. As an alternative, we propose to model the imaging process by decomposing it into a signal decay-independent MR sampling part and an approximation of the signal decay effect. We approximate the latter as a convolution with a Gaussian PSF or, if the effect is that of high-pass filtering, as reversing the effect of a convolution with a Gaussian PSF. We show that for typical high-resolution fMRI at 7 Tesla, signal decay in Spin-Echo has a moderate blurring effect (FWHM = 0.89 voxels, corresponds to 0.44 mm for 0.5 mm wide voxels). In contrast, Gradient-Echo acts as a moderate high-pass filter that can be interpreted as reversing a Gaussian blurring with FWHM = 0.59 voxels (0.30 mm for 0.5 mm wide voxels). Our improved approximations and findings hold not only for Gradient-Echo and Spin-Echo fMRI but also for GRASE and VASO fMRI. Our findings support the correct planning, interpretation, and modeling of high-resolution fMRI. In our first study we used our model to analyze imaging of cortical columns under a very specific scenario. We studied a best case scenario for decoding the stimulated eye from ODCs imaged at 3T using large voxels. In order to do so, we formalized available knowledge about fMRI of cortical columns. In particular, the ability of fMRI to resolve cortical columnar organization depends on several interdependent factors, e.g. the spatial scale of the columnar pattern, the point-spread of the BOLD response, voxel size and the signal-to-noise ratio. In our fourth study we aim to analyze how these factors contribute and combine in imaging of arbitrary cortical columnar patterns at varying field strengths and voxel sizes. In addition, we compared different pattern imaging approaches. We show how detection, decoding and reconstruction of a fine scale organization depend on the parameters of the model, and we predict optimal voxel sizes for each approach under various scenario. The capacity of fMRI to resolve cortical columnar organizations depends on several factors, e.g. the spatial scale of the columnar pattern, the point-spread of the fMRI response, the voxel size, and the SNR considering thermal and physiological noise. How these factors combine, and what is the voxel size that optimizes fMRI of cortical columns remain unknown. Here we combine current knowledge into a quantitative model of fMRI of patterns of cortical columns. We compare different approaches for imaging patterns of cortical columns, including univariate and multivariate based detection, multi-voxel pattern analysis (MVPA) based decoding, and reconstruction of the pattern of cortical columns. We present the dependence of their performance on the parameters of the imaged pattern and the data acquisition, and predict voxel sizes that optimize fMRI under various scenarios. To this end, we modeled differential imaging of realistic patterns of cortical columns with different spatial scales and degrees of irregularity. We quantified the capacity to detect and decode stimulus-specific responses by analyzing the distribution of voxel-wise differential responses relative to noise. We quantified the accuracy with which the spatial pattern of cortical columns can be reconstructed as the correlation between the underlying columnar pattern and the imaged pattern. For regular patterns, optimal voxel widths for detection, decoding and reconstruction were close to half the main cycle length of the organization. Optimal voxel widths for irregular patterns were less dependent on the main cycle length, and differed between univariate detection, multivariate detection and decoding, and reconstruction. We compared the effects of different factors of Gradient Echo fMRI at 3 Tesla (T), Gradient Echo fMRI at 7T and Spin-Echo fMRI at 7T, and found that for all measures (detection, decoding, and reconstruction), the width of the fMRI point-spread has the most significant effect. In contrast, different response amplitudes and noise characteristics played a comparatively minor role. We recommend specific voxel widths for optimal univariate detection, for multivariate detection and decoding, and for reconstruction under these three data-acquisition scenarios. Our study supports the planning, optimization, and interpretation of fMRI of cortical columns and the decoding of information conveyed by these columns

What Makes a Pattern? Matching Decoding Methods to Data in Multivariate Pattern Analysis

Research in neuroscience faces the challenge of integrating information across different spatial scales of brain function. A promising technique for harnessing information at a range of spatial scales is multivariate pattern analysis (MVPA) of functional magnetic resonance imaging (fMRI) data. While the prevalence of MVPA has increased dramatically in recent years, its typical implementations for classification of mental states utilize only a subset of the information encoded in local fMRI signals. We review published studies employing multivariate pattern classification since the technique’s introduction, which reveal an extensive focus on the improved detection power that linear classifiers provide over traditional analysis techniques. We demonstrate using simulations and a searchlight approach, however, that non-linear classifiers are capable of extracting distinct information about interactions within a local region. We conclude that for spatially localized analyses, such as searchlight and region of interest, multiple classification approaches should be compared in order to match fMRI analyses to the properties of local circuits

Mapping the Organization of Axis of Motion Selective Features in Human Area MT Using High-Field fMRI

Functional magnetic resonance imaging (fMRI) at high magnetic fields has made it possible to investigate the columnar organization of the human brain in vivo with high degrees of accuracy and sensitivity. Until now, these results have been limited to the organization principles of early visual cortex (V1). While the middle temporal area (MT) has been the first identified extra-striate visual area shown to exhibit a columnar organization in monkeys, evidence of MT's columnar response properties and topographic layout in humans has remained elusive. Research using various approaches suggests similar response properties as in monkeys but failed to provide direct evidence for direction or axis of motion selectivity in human area MT. By combining state of the art pulse sequence design, high spatial resolution in all three dimensions (0.8 mm isotropic), optimized coil design, ultrahigh field magnets (7 Tesla) and novel high resolution cortical grid sampling analysis tools, we provide the first direct evidence for large-scale axis of motion selective feature organization in human area MT closely matching predictions from topographic columnar-level simulations

Knowing with Which Eye We See: Utrocular Discrimination and Eye-Specific Signals in Human Visual Cortex

Neurophysiological and behavioral reports converge to suggest that monocular neurons in the primary visual cortex are biased toward low spatial frequencies, while binocular neurons favor high spatial frequencies. Here we tested this hypothesis with functional magnetic resonance imaging (fMRI). Human participants viewed flickering gratings at one of two spatial frequencies presented to either the left or the right eye, and judged which of the two eyes was being stimulated (utrocular discrimination). Using multivoxel pattern analysis we found that local spatial patterns of signals in primary visual cortex (V1) allowed successful decoding of the eye-of-origin. Decoding was above chance for low but not high spatial frequencies, confirming the presence of a bias reported by animal studies in human visual cortex. Behaviorally, we found that reliable judgment of the eye-of-origin did not depend on spatial frequency. We further analyzed the mean response in visual cortex to our stimuli and revealed a weak difference between left and right eye stimulation. Our results are thus consistent with the interpretation that participants use overall levels of neural activity in visual cortex, perhaps arising due to local luminance differences, to judge the eye-of-origin. Taken together, we show that it is possible to decode eye-specific voxel pattern information in visual cortex but, at least in healthy participants with normal binocular vision, these patterns are unrelated to awareness of which eye is being stimulated

Spatial scale and distribution of neurovascular signals underlying decoding of orientation and eye of origin from fMRI data

Multivariate pattern analysis of functional magnetic resonance imaging (fMRI) data is widely used, yet the spatial scales and origin of neurovascular signals underlying such analyses remain unclear. We compared decoding performance for stimulus orientation and eye of origin from fMRI measurements in human visual cortex with predictions based on the columnar organization of each feature and estimated the spatial scales of patterns driving decoding. Both orientation and eye of origin could be decoded significantly above chance in early visual areas (V1–V3). Contrary to predictions based on a columnar origin of response biases, decoding performance for eye of origin in V2 and V3 was not significantly lower than that in V1, nor did decoding performance for orientation and eye of origin differ significantly. Instead, response biases for both features showed large-scale organization, evident as a radial bias for orientation, and a nasotemporal bias for eye preference. To determine whether these patterns could drive classification, we quantified the effect on classification performance of binning voxels according to visual field position. Consistent with large-scale biases driving classification, binning by polar angle yielded significantly better decoding performance for orientation than random binning in V1–V3. Similarly, binning by hemifield significantly improved decoding performance for eye of origin. Patterns of orientation and eye preference bias in V2 and V3 showed a substantial degree of spatial correlation with the corresponding patterns in V1, suggesting that response biases in these areas originate in V1. Together, these findings indicate that multivariate classification results need not reflect the underlying columnar organization of neuronal response selectivities in early visual areas. NEW & NOTEWORTHY Large-scale response biases can account for decoding of orientation and eye of origin in human early visual areas V1–V3. For eye of origin this pattern is a nasotemporal bias; for orientation it is a radial bias. Differences in decoding performance across areas and stimulus features are not well predicted by differences in columnar-scale organization of each feature. Large-scale biases in extrastriate areas are spatially correlated with those in V1, suggesting biases originate in primary visual cortex

Emotional Prosody Processing in the Schizophrenia Spectrum.

THESIS ABSTRACT Emotional prosody processing impairment is proposed to be a main contributing factor for the formation of auditory verbal hallucinations in patients with schizophrenia. In order to evaluate such assumption, five experiments in healthy, highly schizotypal and schizophrenia populations are presented. The first part of the thesis seeks to reveal the neural underpinnings of emotional prosody comprehension (EPC) in a non-clinical population as well as the modulation of prosodic abilities by hallucination traits. By revealing the brain representation of EPC, an overlap at the neural level between EPC and auditory verbal hallucinations (AVH) was strongly suggested. By assessing the influence of hallucinatory traits on EPC abilities, a continuum in the schizophrenia spectrum in which high schizotypal population mirrors the neurocognitive profile of schizophrenia patients was established. Moreover, by studying the relation between AVH and EPC in non-clinical population, potential confounding effects of medication influencing the findings were minimized. The second part of the thesis assessed two EPC related abilities in schizophrenia patients with and without hallucinations. Firstly, voice identity recognition, a skill which relies on the analysis of some of the same acoustical features as EPC, has been evaluated in patients and controls. Finally, the last study presented in the current thesis, assessed the influence that implicit processing of emotional prosody has on selective attention in patients and controls. Both patients studies demonstrate that voice identity recognition deficits as well as abnormal modulation of selective attention by implicit emotion prosody are related to hallucinations exclusively and not to schizophrenia in general. In the final discussion, a model in which EPC deficits are a crucial factor in the formation of AVH is evaluated. Experimental findings presented in the previous chapters strongly suggests that the perception of prosodic features is impaired in patients with AVH, resulting in aberrant perception of irrelevant auditory objects with emotional prosody salience which captures the attention of the hearer and which sources (speaker identity) cannot be recognized. Such impairments may be due to structural and functional abnormalities in a network which comprises the superior temporal gyrus as a central element

Direct evidence for encoding of motion streaks in human visual cortex

Temporal integration in the visual system causes fast-moving objects to generate static, oriented traces ('motion streaks'), which could be used to help judge direction of motion. While human psychophysics and single-unit studies in non-human primates are consistent with this hypothesis, direct neural evidence from the human cortex is still lacking. First, we provide psychophysical evidence that faster and slower motions are processed by distinct neural mechanisms: faster motion raised human perceptual thresholds for static orientations parallel to the direction of motion, whereas slower motion raised thresholds for orthogonal orientations. We then used functional magnetic resonance imaging to measure brain activity while human observers viewed either fast ('streaky') or slow random dot stimuli moving in different directions, or corresponding static-oriented stimuli. We found that local spatial patterns of brain activity in early retinotopic visual cortex reliably distinguished between static orientations. Critically, a multivariate pattern classifier trained on brain activity evoked by these static stimuli could then successfully distinguish the direction of fast ('streaky') but not slow motion. Thus, signals encoding static-oriented streak information are present in human early visual cortex when viewing fast motion. These experiments show that motion streaks are present in the human visual system for faster motion.This work was supported by the Wellcome Trust (G.R., D.S.S.), the European Union ‘Mindbridge’ project (B.B.), the Australian Federation of Graduate Women Tempe Mann Scholarship (D.A.), the University of Sydney Campbell Perry Travel Fellowship (D.A.) and the Brain Research Trust (C.K.)

Fast, invariant representation for human action in the visual system

Humans can effortlessly recognize others' actions in the presence of complex transformations, such as changes in viewpoint. Several studies have located the regions in the brain involved in invariant action recognition, however, the underlying neural computations remain poorly understood. We use magnetoencephalography (MEG) decoding and a dataset of well-controlled, naturalistic videos of five actions (run, walk, jump, eat, drink) performed by different actors at different viewpoints to study the computational steps used to recognize actions across complex transformations. In particular, we ask when the brain discounts changes in 3D viewpoint relative to when it initially discriminates between actions. We measure the latency difference between invariant and non-invariant action decoding when subjects view full videos as well as form-depleted and motion-depleted stimuli. Our results show no difference in decoding latency or temporal profile between invariant and non-invariant action recognition in full videos. However, when either form or motion information is removed from the stimulus set, we observe a decrease and delay in invariant action decoding. Our results suggest that the brain recognizes actions and builds invariance to complex transformations at the same time, and that both form and motion information are crucial for fast, invariant action recognition