On the geometric structure of fMRI searchlight-based information maps

Information mapping is a popular application of Multivoxel Pattern Analysis (MVPA) to fMRI. Information maps are constructed using the so called searchlight method, where the spherical multivoxel neighborhood of every voxel (i.e., a searchlight) in the brain is evaluated for the presence of task-relevant response patterns. Despite their widespread use, information maps present several challenges for interpretation. One such challenge has to do with inferring the size and shape of a multivoxel pattern from its signature on the information map. To address this issue, we formally examined the geometric basis of this mapping relationship. Based on geometric considerations, we show how and why small patterns (i.e., having smaller spatial extents) can produce a larger signature on the information map as compared to large patterns, independent of the size of the searchlight radius. Furthermore, we show that the number of informative searchlights over the brain increase as a function of searchlight radius, even in the complete absence of any multivariate response patterns. These properties are unrelated to the statistical capabilities of the pattern-analysis algorithms used but are obligatory geometric properties arising from using the searchlight procedure.Comment: 15 pages, 7 figure

Neural Univariate Activity and Multivariate Pattern in the Posterior Superior Temporal Sulcus Differentially Encode Facial Expression and Identity

Faces contain a variety of information such as one’s identity and expression. One prevailing model suggests a functional division of labor in processing faces that different aspects of facial information are processed in anatomically separated and functionally encapsulated brain regions. Here, we demonstrate that facial identity and expression can be processed in the same region, yet with different neural coding strategies. To this end, we employed functional magnetic resonance imaging to examine two types of coding schemes, namely univariate activity and multivariate pattern, in the posterior superior temporal cortex (pSTS) - a face-selective region that is traditionally viewed as being specialized for processing facial expression. With the individual difference approach, we found that participants with higher overall face selectivity in the right pSTS were better at differentiating facial expressions measured outside of the scanner. In contrast, individuals whose spatial pattern for faces in the right pSTS was less similar to that for objects were more accurate in identifying previously presented faces. The double dissociation of behavioral relevance between overall neural activity and spatial neural pattern suggests that the functional-division-of-labor model on face processing is over-simplified, and that coding strategies shall be incorporated in a revised model

A toolbox for representational similarity analysis.

Neuronal population codes are increasingly being investigated with multivariate pattern-information analyses. A key challenge is to use measured brain-activity patterns to test computational models of brain information processing. One approach to this problem is representational similarity analysis (RSA), which characterizes a representation in a brain or computational model by the distance matrix of the response patterns elicited by a set of stimuli. The representational distance matrix encapsulates what distinctions between stimuli are emphasized and what distinctions are de-emphasized in the representation. A model is tested by comparing the representational distance matrix it predicts to that of a measured brain region. RSA also enables us to compare representations between stages of processing within a given brain or model, between brain and behavioral data, and between individuals and species. Here, we introduce a Matlab toolbox for RSA. The toolbox supports an analysis approach that is simultaneously data- and hypothesis-driven. It is designed to help integrate a wide range of computational models into the analysis of multichannel brain-activity measurements as provided by modern functional imaging and neuronal recording techniques. Tools for visualization and inference enable the user to relate sets of models to sets of brain regions and to statistically test and compare the models using nonparametric inference methods. The toolbox supports searchlight-based RSA, to continuously map a measured brain volume in search of a neuronal population code with a specific geometry. Finally, we introduce the linear-discriminant t value as a measure of representational discriminability that bridges the gap between linear decoding analyses and RSA. In order to demonstrate the capabilities of the toolbox, we apply it to both simulated and real fMRI data. The key functions are equally applicable to other modalities of brain-activity measurement. The toolbox is freely available to the community under an open-source license agreement (http://www.mrc-cbu.cam.ac.uk/methods-and-resources/toolboxes/license/)

a methodological approach

In natural environments, visual and auditory stimulation elicit responses across a large set of brain regions in a fraction of a second, yielding representations of the multimodal scene and its properties. The rapid and complex neural dynamics underlying visual and auditory information processing pose major challenges to human cognitive neuroscience. Brain signals measured non-invasively are inherently noisy, the format of neural representations is unknown, and transformations between representations are complex and often nonlinear. Further, no single non-invasive brain measurement technique provides a spatio-temporally integrated view. In this opinion piece, we argue that progress can be made by a concerted effort based on three pillars of recent methodological development: (i) sensitive analysis techniques such as decoding and cross-classification, (ii) complex computational modelling using models such as deep neural networks, and (iii) integration across imaging methods (magnetoencephalography/electroencephalography, functional magnetic resonance imaging) and models, e.g. using representational similarity analysis. We showcase two recent efforts that have been undertaken in this spirit and provide novel results about visual and auditory scene analysis. Finally, we discuss the limits of this perspective and sketch a concrete roadmap for future research

FMRI and ERP investigations of body representations in the human lateral and ventral occipitotemporal cortex

EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Object processing in the medial temporal lobe: Influence of object domain

We live in a rich visual world, surrounded by many different kinds of objects. While we may not often reflect on it, our ability to recognize what an object is, detect whether an object is familiar or novel, and bring to mind our general knowledge about an object, are all essential components of adaptive behavior. In this dissertation, I investigate the neural basis of object representations, focusing on medial temporal lobe (MTL) structures, namely, perirhinal cortex, parahippocampal cortex, and hippocampus. I use what type of thing an object is, or more specifically, the broader category (e.g., “face” or “house”) or domain (e.g., “animate or “inanimate”) to which an object belongs to probe MTL structures. In the Chapter 2, I used fMRI to explore whether object representations in MTL structures were organized by animacy, and/or real-world size. I found domain-level organization in all three MTL structures, with a distinct pattern of domain organization in each structure. In Chapter 3, I examined whether recognition-memory signals for objects were organized by category and domain in the same MTL structures. I found no evidence of category or domain specificity in recognition memory-signals, but did reveal a distinction between novel and familiar object representations across all categories. Finally, in Chapter 4, I used a neuropsychological approach to discover a unique contribution of the hippocampus to object concepts. I found that an individual with developmental amnesia had normal intrinsic feature knowledge, but less extrinsic, or associative feature knowledge of concepts This decreased extrinsic feature knowledge led to abnormalities specific to non-living object concepts. These results show that the hippocampus may play an important role in the development of object concepts, potentially through the same relational binding mechanism that links objects and context in episodic memory. Taken together, these findings suggest that using object category or domain to probe the function of MTL structures is a useful approach for gaining a deeper understanding of the similarities and differences between MTL structures, and how they contribute more broadly to our perception and memory of the world

Kuuloaivokuoren toiminnallinen organisaatio aktiivisten kuuntelutehtävien aikana

Previous imaging studies have shown that activation in human auditory cortex (AC) is strongly modulated during active listening tasks. However, the prevalent models of AC mainly focus on the processing of stimulus-specific information and speech and do not predict such task-dependent modulation. In the present thesis, functional magnetic resonance imaging was used to measure regional activation in AC during discrimination and n-back memory tasks in order to investigate the relationship between stimulus-specific and task-dependent processing (Study I) and inter-regional connectivity during rest and active tasks (Study III). In addition, source analysis of scalp-recorded event-related potentials was carried out to study the temporal dynamics of task-dependent activation in AC (Study II). In Study I, distinct stimulus-specific activation patterns to pitch-varying and location-varying sounds were similarly observed during visual (no directed auditory attention) and auditory tasks. This is consistent with the prevalent models which presume parallel and independent “what” (e.g. pitch) and “where” processing streams. As expected, discrimination and n-back memory tasks were associated with distinct task-dependent activation patterns. These activation patterns were independent of whether subjects performed pitch or location versions of these tasks. Thus, AC activation during discrimination and n-back memory tasks cannot be explained by enhanced stimulus-specific processing (of pitch and location). Consistently, Study II showed that the task-dependent effects in AC occur relatively late (200–700 ms from stimulus onset) compared to the latency of stimulus-specific pitch processing (0–200 ms). In Study III, the organization of human AC was investigated based on functional connectivity. Connectivity-based parcellation revealed a network structure that consisted of six modules in supratemporal plane, temporal lobe, and inferior parietal lobule in both hemispheres. Multivariate pattern analysis showed that connectivity within this network structure was significantly modulated during the presentation of sounds (visual task) and auditory task performance. Together the results of this thesis show that (1) activation in human AC strongly depends on the requirements of the listening task and that task-dependent modulation is not due to enhanced stimulus-specific processing, (2) regions in inferior parietal lobule play an important role in the processing of both task-irrelevant and task-relevant auditory information in human AC, and (3) the activation patterns in human AC during the presentation of task-irrelevant and task-relevant sounds cannot be fully explained by a hierarchical model in which information is processed in two parallel processing streams.Aiemmat kuvantamistutkimukset ovat osoittaneet, että aktiiviset kuuntelutehtävät vaikuttavat voimakkaasti ihmisen kuuloaivokuoren aktivaatioon. Kuuloaivokuoren toiminnalliset mallit kuitenkin keskittyvät äänten akustisten piirteiden ja puheen käsittelyyn, eivätkä ne siten ennusta tehtäväsidonnaisia vaikutuksia. Tässä väitöskirjassa tutkittiin kuuloaivokuoren toimintaa äänten erottelu- ja n-back-muistitehtävien aikana toiminnallisella magneettikuvauksella ja herätevasterekisteröinnillä. Tutkimusten tavoitteena oli selvittää riippuvatko ärsyke- ja tehtäväsidonnaiset aktivaatiot toisistaan (Tutkimus I) sekä tutkia kuuloaivokuoren eri alueiden välistä toiminnallista konnektiivisuutta lepo- ja tehtävätilanteiden aikana (Tutkimus III). Pään pinnalta mitattujen herätevasteiden lähdemallinnuksen avulla tutkittiin kuuloaivokuoren tehtäväsidonnaisen aktivaation ajallista dynamiikkaa (Tutkimus II). Tutkimuksessa I äänen korkeuden ja tulosuunnan vaihtelu aktivoivat erillisiä kuuloaivokuoren alueita sekä näkötehtävän (ei suunnattua kuulotarkkaavaisuutta) että kuuntelutehtävien aikana. Tämä tulos on yhtenevä vallitsevien kuuloaivokuoren mallien kanssa, joissa oletetaan, että äänen korkeus ja tulosuunta käsitellään rinnakkaisissa ja toisistaan riippumattomissa mitä- (esim. äänen korkeus) ja missä-järjestelmissä. Aktiivisten kuuntelutehtävien aikana kuuloaivokuoren aktivaatiojakauma riippui odotetusti siitä, tekivätkö koehenkilöt äänten erottelu vai n-back-muistitehtävää. Tehtäväsidonnaiset aktivaatiojakaumat (erottelu- ja muistitehtävän erot) olivat kuitenkin hyvin samankaltaisia äänen korkeus- ja tulosuuntatehtävien aikana. Kuuloaivokuoren tehtäväsidonnaisia aktivaatioita äänten erottelu- ja n-back-muistitehtävien aikana ei siten voida selittää ääni-informaation käsittelyyn liittyvien aktivaatioiden voimistumisella. Tätä johtopäätöstä tukevat myös Tutkimuksen II tulokset, joiden mukaan kuuloaivokuoren tehtäväsidonnaiset aktivaatiot (n. 200–700 ms äänen alusta) havaitaan pääosin äänenkorkeustiedon käsittelyyn liittyvän aktivaation (0–200 ms) jälkeen. Tutkimuksessa III selvitettiin kuuloaivokuoren toiminnallista organisaatiota ja sen eri aluiden muodostamia verkostoja konnektiivisuusanalyysien avulla. Näissä analyyseissä havaittiin modulaarinen rakenne, jossa kuuloaivokuori ja sen lähialueiden muodostama verkosto jakaantuu kuuteen osaan (moduuliin). Toiminnallisen konnektiivisuuden muutoksia eri koetilanteissa tarkasteltiin monimuuttujakuvioanalyysillä. Tulokset osoittivat, että konnektiivisuus kuuloaivokuoren ja sen lähialueiden muodostamassa verkostossa muuttui merkitsevästi verrattuna lepotilanteeseen, kun koehenkilöille esitettiin ääniä (näkötehtävän aikana) ja kun he tekivät kuuntelutehtäviä. Väitöskirjan tutkimusten tulokset osoittavat, että (1) ihmisen kuuloaivokuoren aktivaatio riippuu voimakkaasti kuuntelutehtävän vaatimuksista, (2) päälaenlohkon alaosat osallistuvat merkittävällä tavalla kuuloaivokuoren toimintaan ääni-informaation käsittelyn aikana riippumatta siitä, ovatko äänet olennaisia vai epäolennaisia kulloisenkin tehtävän kannalta, ja (3) ihmisen kuuloaivokuoren aktivaatiota ei voida täysin selittää hierarkisella mallilla, jossa ääni-informaatiota käsitellään kahdella rinnakkaisella tiedonkäsittelyradalla

Modeling and analysis of mechanisms underlying high-resolution functional MRI of cortical columns

High spatial resolution functional MRI (fMRI) and advanced multivariate analysis techniques are promising tools for studying the cortical basis of human cognitive processes at the level of columns and layers. However the true spatial specificity of high-resolution fMRI has not been quantified, and the basis for decoding from fine scale structures using large voxels and relatively low magnetic field strength is unknown. It is also not yet known what method and voxel size is optimal for decoding and what voxel size is optimal for high-resolution imaging. In this thesis we present four studies that answer part of these questions using a model-based approach of imaging cortical columns. We started our investigation of model-based analysis of high-resolution fMRI of cortical columns by addressing the specific problem of how it is possible to decode information thought to be mediated by cortical columns using large voxels at low field strength. Multivariate machine learning algorithms applied to human functional MRI (fMRI) data can decode information conveyed by cortical columns, despite the voxel-size being large relative to the width of columns. Several mechanisms have been proposed to underlie decoding of stimulus orientation or the stimulated eye. These include: (I) aliasing of high spatial-frequency components, including the main frequency component of the columnar organization, (II) contributions from local irregularities in the columnar organization, (III) contributions from large-scale non-columnar organizations, (IV) functionally selective veins with biased draining regions, and (V) complex spatio-temporal filtering of neuronal activity by fMRI voxels. Here we sought to assess the plausibility of two of the suggested mechanisms: (I) aliasing and (II) local irregularities, using a naive model of BOLD as blurring and MRI voxel sampling. To this end, we formulated a mathematical model that encompasses both the processes of imaging ocular dominance (OD) columns and the subsequent linear classification analysis. Through numerical simulations of the model, we evaluated the distribution of functional differential contrasts that can be expected when considering the pattern of cortical columns, the hemodynamic point spread function, the voxel size, and the noise. We found that with data acquisition parameters used at 3 Tesla, sub-voxel supra-Nyquist frequencies, including frequencies near the main frequency of the OD organization (0.5 cycles per mm), cannot contribute to the differential contrast. The differential functional contrast of local origin is dominated by low-amplitude contributions from low frequencies, associated with irregularities of the cortical pattern. Realizations of the model with parameters that reflected a best-case scenario and the reported BOLD point-spread at 3 Tesla (3.5 mm) predicted decoding performances lower than those that have been previously obtained at this magnetic field strength. We conclude that low frequency components that underlie local irregularities in the columnar organization are likely to play a role in decoding. We further expect that fMRI-based decoding relies, in part, on signal contributions from large-scale, non-columnar functional organizations, and from complex spatio-temporal filtering of neuronal activity by fMRI voxels, involving biased venous responses. Our model can potentially be used for evaluating and optimizing data-acquisition parameters for decoding information conveyed by cortical columns. Having developed a model of imaging ODCs we then used this model to estimate the spatial specificity of BOLD fMRI, specifically at high field (7 T). Previous attempts at characterizing the spatial specificity of the blood oxygenation level dependent functional MRI (BOLD fMRI) response by estimating its point-spread function (PSF) have conventionally relied on spatial representations of visual stimuli in area V1. Consequently, their estimates were confounded by the width and scatter of receptive fields of V1 neurons. Here, we circumvent these limits by instead using the inherent cortical spatial organization of ocular dominance columns (ODCs) to determine the PSF for both Gradient Echo (GE) and Spin Echo (SE) BOLD imaging at 7 Tesla. By applying Markov Chain Monte Carlo sampling on a probabilistic generative model of imaging ODCs, we quantified the PSFs that best predict the spatial structure and magnitude of differential ODCs’ responses. Prior distributions for the ODC model parameters were determined by analyzing published data of cytochrome oxidase patterns from post-mortem histology of human V1 and of neurophysiological ocular dominance indices. The most probable PSF full-widths at half-maximum were 0.82 mm (SE) and 1.02 mm (GE). Our results provide a quantitative basis for the spatial specificity of BOLD fMRI at ultra-high fields, which can be used for planning and interpretation of high-resolution differential fMRI of fine-scale cortical organizations. Our BOLD fMRI PSF findings show that the PSF is considerably smaller than what was reported previously. This in turn raised the question of the role of the imaging PSF, which now has become relevant. Next, we show that the commonly used magnitude point-spread function fails to accurately represent the true effects of k-space sampling and signal decay, and propose an alternative model that accounts more accurately for these effects. The effects of k-space sampling and signal decay on the effective spatial resolution of MRI and functional MRI (fMRI) are commonly assessed by means of the magnitude point-spread function (PSF), defined as the absolute values (magnitudes) of the complex MR imaging PSF. It is commonly assumed that this magnitude PSF signifies blurring, which can be quantified by its full-width at half-maximum (FWHM). Here we show that the magnitude PSF fails to accurately represent the true effects of k-space sampling and signal decay. Firstly, a substantial part of the width of the magnitude PSF is due to MRI sampling per se. This part is independent of any signal decay and its effect depends on the spatial frequency composition of the imaged object. Therefore, it cannot always be expected to introduce blurring. Secondly, MRI reconstruction is typically followed by taking the absolute values (magnitude image) of the reconstructed complex image. This introduces a non-linear stage into the process of image formation. The complex imaging PSF does not fully describe this process, since it does not reflect the stage of taking the magnitude image. Its corresponding magnitude PSF fails to correctly describe this process, since convolving the original pattern with the magnitude PSF is different from the true process of taking the absolute following a convolution with the complex imaging PSF. Lastly, signal decay can have not only a blurring, but also a high-pass filtering effect. This cannot be reflected by the strictly positive width of the magnitude PSF. As an alternative, we propose to model the imaging process by decomposing it into a signal decay-independent MR sampling part and an approximation of the signal decay effect. We approximate the latter as a convolution with a Gaussian PSF or, if the effect is that of high-pass filtering, as reversing the effect of a convolution with a Gaussian PSF. We show that for typical high-resolution fMRI at 7 Tesla, signal decay in Spin-Echo has a moderate blurring effect (FWHM = 0.89 voxels, corresponds to 0.44 mm for 0.5 mm wide voxels). In contrast, Gradient-Echo acts as a moderate high-pass filter that can be interpreted as reversing a Gaussian blurring with FWHM = 0.59 voxels (0.30 mm for 0.5 mm wide voxels). Our improved approximations and findings hold not only for Gradient-Echo and Spin-Echo fMRI but also for GRASE and VASO fMRI. Our findings support the correct planning, interpretation, and modeling of high-resolution fMRI. In our first study we used our model to analyze imaging of cortical columns under a very specific scenario. We studied a best case scenario for decoding the stimulated eye from ODCs imaged at 3T using large voxels. In order to do so, we formalized available knowledge about fMRI of cortical columns. In particular, the ability of fMRI to resolve cortical columnar organization depends on several interdependent factors, e.g. the spatial scale of the columnar pattern, the point-spread of the BOLD response, voxel size and the signal-to-noise ratio. In our fourth study we aim to analyze how these factors contribute and combine in imaging of arbitrary cortical columnar patterns at varying field strengths and voxel sizes. In addition, we compared different pattern imaging approaches. We show how detection, decoding and reconstruction of a fine scale organization depend on the parameters of the model, and we predict optimal voxel sizes for each approach under various scenario. The capacity of fMRI to resolve cortical columnar organizations depends on several factors, e.g. the spatial scale of the columnar pattern, the point-spread of the fMRI response, the voxel size, and the SNR considering thermal and physiological noise. How these factors combine, and what is the voxel size that optimizes fMRI of cortical columns remain unknown. Here we combine current knowledge into a quantitative model of fMRI of patterns of cortical columns. We compare different approaches for imaging patterns of cortical columns, including univariate and multivariate based detection, multi-voxel pattern analysis (MVPA) based decoding, and reconstruction of the pattern of cortical columns. We present the dependence of their performance on the parameters of the imaged pattern and the data acquisition, and predict voxel sizes that optimize fMRI under various scenarios. To this end, we modeled differential imaging of realistic patterns of cortical columns with different spatial scales and degrees of irregularity. We quantified the capacity to detect and decode stimulus-specific responses by analyzing the distribution of voxel-wise differential responses relative to noise. We quantified the accuracy with which the spatial pattern of cortical columns can be reconstructed as the correlation between the underlying columnar pattern and the imaged pattern. For regular patterns, optimal voxel widths for detection, decoding and reconstruction were close to half the main cycle length of the organization. Optimal voxel widths for irregular patterns were less dependent on the main cycle length, and differed between univariate detection, multivariate detection and decoding, and reconstruction. We compared the effects of different factors of Gradient Echo fMRI at 3 Tesla (T), Gradient Echo fMRI at 7T and Spin-Echo fMRI at 7T, and found that for all measures (detection, decoding, and reconstruction), the width of the fMRI point-spread has the most significant effect. In contrast, different response amplitudes and noise characteristics played a comparatively minor role. We recommend specific voxel widths for optimal univariate detection, for multivariate detection and decoding, and for reconstruction under these three data-acquisition scenarios. Our study supports the planning, optimization, and interpretation of fMRI of cortical columns and the decoding of information conveyed by these columns