Coordinate based random effect size meta-analysis of neuroimaging studies

Low power in neuroimaging studies can make them difficult to interpret, and Coordinate based meta-analysis (CBMA) may go some way to mitigating this issue. CBMA has been used in many analyses to detect where published functional MRI or voxel-based morphometry studies testing similar hypotheses report significant summary results (coordinates) consistently. Only the reported coordinates and possibly t statistics are analysed, and statistical significance of clusters is determined by coordinate density. Here a method of performing coordinate based random effect size meta-analysis and meta-regression is introduced. The algorithm (ClusterZ) analyses both coordinates and reported t statistic or Z score, standardised by the number of subjects. Statistical significance is determined not by coordinate density, but by a random effects meta-analyses of reported effects performed cluster-wise using standard statistical methods and taking account of censoring inherent in the published summary results. Type 1 error control is achieved using the false cluster discovery rate (FCDR), which is based on the false discovery rate. This controls both the family wise error rate under the null hypothesis that coordinates are randomly drawn from a standard stereotaxic space, and the proportion of significant clusters that are expected under the null. Such control is necessary to avoid propagating and even amplifying the very issues motivating the meta-analysis in the first place. ClusterZ is demonstrated on both numerically simulated data and on real data from reports of grey matter loss in multiple sclerosis (MS) and syndromes suggestive of MS, and of painful stimulus in healthy controls. The software implementation is available to download and use freely

Assessing publication bias in coordinate-based meta-analysis techniques?

Introduction While publications of fMRI studies have flourished, it is increasingly recognized that progress in understanding human brain function will require integration of data across studies using meta-analyses. In general, results that do not reach statistical significance are less likely to be published and included in a meta-analysis. Meta-analyses of fMRI studies are prone to this publication bias when studies are excluded because they fail to show activation in specific regions. Further, some studies only report a limited amount of peak voxels that survive a statistical threshold resulting in an enormous loss of data. Coordinate-based toolboxes have been specifically developed to combine the available information of such studies in a meta-analysis. Potential publication bias then stems from two sources: exclusion of studies and missing voxel information within studies. In this study, we focus on the assessment of the first source of bias in coordinate-based meta-analyses. A measure of publication bias indicates the degree to which the analysis might be distorted and helps to interpret results. We propose an adaptation of the Fail-Safe N (FSN; Rosenthal, 1979). The FSN reflects the number of null studies, i.e. studies without activation in a target region, that can be added to an existing meta-analysis without altering the result for the target region. A large FSN indicates robustness of the effect against publication bias. On the other hand, in this context, a FSN that is too large indicates that a small amount of studies might drive the entire analysis. Method We simulated 1000 simplistic meta-analyses, each consisting of 3 studies with real activation in a target area (quadrant 1 in Figure 1) and up to 100 null studies with activation in the remaining 3 quadrants. We calculated the FSN as the number of null studies (with a maximum of 100) that can be added to the original meta-analysis of 3 studies without altering the results for the target area. Meta-analyses were conducted with ALE (Eickhoff et al., 2009; 2012; Turkeltaub et al., 2012).  We computed the FSN using an uncorrected threshold (α = 0.001) and 2 versions of a False Discovery Rate (FDR) threshold (q = 0.05), FDR pID (which assumes independence or positive dependence between test statistics) and FDR pN (which makes no assumptions and is more conservative). We varied the average sample size n of the individual studies, from small (n≈10), to medium (n≈20) and large (n≈30). Results Results are summarised in Figure 2 and visually presented in Figure 3. We find a large difference in average FSN between the different thresholding methods. In case of uncorrected thresholding, the target region remains labeled as active while only 3% of the studies in the meta-analysis report activation at that location.  Further, if the sample size of the individual studies in the meta-analysis increases, the FSN decreases. Conclusions The FSN varies largely across thresholding methods and sample sizes. Uncorrected thresholding allows for the analysis to be driven by a small amount of studies and is therefore counter-indicated. While a decreasing FSN with increasing sample size might be counterintuitive in terms of robustness, it indicates that the analysis is less prone to be driven by a small number of studies. Publication bias assessment methods can be a valuable add-on to existing toolboxes for interpretation of meta-analytic results. In future work, we will extend our research to other methods for the assessment of publication bias, such as the Egger Test  (Egger et al., 1997) and test for excess of success (Francis, 2014). References Egger, M., Davey Smith, G., Schneider, M., and Minder, C. (1997), ‘Bias in meta-analysis detected by a simple, graphical test’, British Medical Journal, vol. 315, pp. 629-634. Eickhoff, S.B., Laird, A.R., Grefkes, C., Wang, L.E., Zilles, K., and Fox, P.T. (2009), ‘Coordinate-based activation likelihood estimation meta-analysis of neuroimaging data: A random-effects approach based on empirical estimates of spatial uncertainty’, Human Brain Mapping, vol. 30, pp. 2907-2926. Eickhoff, S.B., Bzdok, D., Laird, A.R., Kurth, F., and Fox, P.T. (2012), ‘Activation likelihood estimation revisited’, Neuroimage, vol. 59, pp. 2349-2361. Francis, G. (2014), ‘The frequency of excess success for articles in Psychological Science’, Psychonomic Bulletin and Review, vol. 21, no. 5, pp. 1180-1187. Rosenthal, R. (1979), ‘The file drawer problem and tolerance for null results’, Psychological Bulletin, vol. 86, no. 3, pp. 638–641. Turkeltaub, P.E., Eickhoff, S.B., Laird, A.R., Fox, M., Wiener, M., and Fox, P. (2012), ‘Minimizing within-experiment and within-group effects in activation likelihood estimation meta-analyses’, Human Brain Mapping, vol. 33, pp. 1-13

Meta-analysis of functional neuroimaging data using Bayesian nonparametric binary regression

In this work we perform a meta-analysis of neuroimaging data, consisting of locations of peak activations identified in 162 separate studies on emotion. Neuroimaging meta-analyses are typically performed using kernel-based methods. However, these methods require the width of the kernel to be set a priori and to be constant across the brain. To address these issues, we propose a fully Bayesian nonparametric binary regression method to perform neuroimaging meta-analyses. In our method, each location (or voxel) has a probability of being a peak activation, and the corresponding probability function is based on a spatially adaptive Gaussian Markov random field (GMRF). We also include parameters in the model to robustify the procedure against miscoding of the voxel response. Posterior inference is implemented using efficient MCMC algorithms extended from those introduced in Holmes and Held [Bayesian Anal. 1 (2006) 145--168]. Our method allows the probability function to be locally adaptive with respect to the covariates, that is, to be smooth in one region of the covariate space and wiggly or even discontinuous in another. Posterior miscoding probabilities for each of the identified voxels can also be obtained, identifying voxels that may have been falsely classified as being activated. Simulation studies and application to the emotion neuroimaging data indicate that our method is superior to standard kernel-based methods.Comment: Published in at http://dx.doi.org/10.1214/11-AOAS523 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Is there a neuroanatomical basis of the vulnerability to suicidal behavior?: a coordinate-based meta-analysis of structural and functional MRI studies

Objective: We conducted meta-analyses of functional and structural neuroimaging studies comparing adolescent and adult individuals with a history of suicidal behavior and a psychiatric disorder to psychiatric controls in order to objectify changes in brain structure and function in association with a vulnerability to suicidal behavior. Methods: Magnetic resonance imaging studies published up to July 2013 investigating structural or functional brain correlates of suicidal behavior were identified through computerized and manual literature searches. Activation foci from 12 studies encompassing 475 individuals, i.e., 213 suicide attempters and 262 psychiatric controls were subjected to meta-analytical study using anatomic or activation likelihood estimation (ALE). Result: Activation likelihood estimation revealed structural deficits and functional changes in association with a history of suicidal behavior. Structural findings included reduced volumes of the rectal gyrus, superior temporal gyrus and caudate nucleus. Functional differences between study groups included an increased reactivity of the anterior and posterior cingulate cortices. Discussion: A history of suicidal behavior appears to be associated with (probably interrelated) structural deficits and functional overactivation in brain areas, which contribute to a decision-making network. The findings suggest that a vulnerability to suicidal behavior can be defined in terms of a reduced motivational control over the intentional behavioral reaction to salient negative stimuli

Bayesian log-Gaussian Cox process regression: applications to meta-analysis of neuroimaging working memory studies

Working memory (WM) was one of the first cognitive processes studied with functional magnetic resonance imaging. With now over 20 years of studies on WM, each study with tiny sample sizes, there is a need for meta-analysis to identify the brain regions that are consistently activated by WM tasks, and to understand the interstudy variation in those activations. However, current methods in the field cannot fully account for the spatial nature of neuroimaging meta-analysis data or the heterogeneity observed among WM studies. In this work, we propose a fully Bayesian random-effects metaregression model based on log-Gaussian Cox processes, which can be used for meta-analysis of neuroimaging studies. An efficient Markov chain Monte Carlo scheme for posterior simulations is presented which makes use of some recent advances in parallel computing using graphics processing units. Application of the proposed model to a real data set provides valuable insights regarding the function of the WM

Effects of cue focality on the neural mechanisms of prospective memory: A meta-analysis of neuroimaging studies

Remembering to execute pre-defined intentions at the appropriate time in the future is typically referred to as Prospective Memory (PM). Studies of PM showed that distinct cognitive processes underlie the execution of delayed intentions depending on whether the cue associated with such intentions is focal to ongoing activity processing or not (i.e., cue focality). The present activation likelihood estimation (ALE) meta-analysis revealed several differences in brain activity as a function of focality of the PM cue. The retrieval of intention is supported mainly by left anterior prefrontal cortex (Brodmann Area, BA 10) in nonfocal tasks, and by cerebellum and ventral parietal regions in focal tasks. Furthermore, the precuneus showed increased activation during the maintenance phase of intentions compared to the retrieval phase in nonfocal tasks, whereas the inferior parietal lobule showed increased activation during the retrieval of intention compared to maintenance phase in the focal tasks. Finally, the retrieval of intention relies more on the activity in anterior cingulate cortex for nonfocal tasks, and on posterior cingulate cortex for focal tasks. Such focality-related pattern of activations suggests that prospective remembering is mediated mainly by top-down and stimulus-independent processes in nonfocal tasks, whereas by more automatic, bottom-up, processes in focal tasks

The meditative mind: a comprehensive meta-analysis of MRI studies

Over the past decade mind and body practices, such as yoga and meditation, have raised interest in different scientific fields; in particular, the physiological mechanisms underlying the beneficial effects observed in meditators have been investigated. Neuroimaging studies have studied the effects of meditation on brain structure and function and findings have helped clarify the biological underpinnings of the positive effects of meditation practice and the possible integration of this technique in standard therapy. The large amount of data collected thus far allows drawing some conclusions about the neural effects of meditation practice. In the present study we used activation likelihood estimation (ALE) analysis to make a coordinate-based meta-analysis of neuroimaging data on the effects of meditation on brain structure and function. Results indicate that meditation leads to activation in brain areas involved in processing self-relevant information, self-regulation, focused problem-solving, adaptive behavior, and interoception. Results also show that meditation practice induces functional and structural brain modifications in expert meditators, especially in areas involved in self-referential processes such as self-awareness and self-regulation. These results demonstrate that a biological substrate underlies the positive pervasive effect of meditation practice and suggest that meditation techniques could be adopted in clinical populations and to prevent disease

The empirical replicability of task-based fMRI as a function of sample size

Replicating results (i.e. obtaining consistent results using a new independent dataset) is an essential part of good science. As replicability has consequences for theories derived from empirical studies, it is of utmost importance to better understand the underlying mechanisms influencing it. A popular tool for non-invasive neuroimaging studies is functional magnetic resonance imaging (fMRI). While the effect of underpowered studies is well documented, the empirical assessment of the interplay between sample size and replicability of results for task-based fMRI studies remains limited. In this work, we extend existing work on this assessment in two ways. Firstly, we use a large database of 1400 subjects performing four types of tasks from the IMAGEN project to subsample a series of independent samples of increasing size. Secondly, replicability is evaluated using a multi-dimensional framework consisting of 3 different measures: (un)conditional test-retest reliability, coherence and stability. We demonstrate not only a positive effect of sample size, but also a trade-off between spatial resolution and replicability. When replicability is assessed voxelwise or when observing small areas of activation, a larger sample size than typically used in fMRI is required to replicate results. On the other hand, when focussing on clusters of voxels, we observe a higher replicability. In addition, we observe variability in the size of clusters of activation between experimental paradigms or contrasts of parameter estimates within these

Behavior, sensitivity, and power of activation likelihood estimation characterized by massive empirical simulation

Given the increasing number of neuroimaging publications, the automated knowledge extraction on brain-behavior associations by quantitative meta-analyses has become a highly important and rapidly growing field of research. Among several methods to perform coordinate-based neuroimaging meta-analyses, Activation Likelihood Estimation (ALE) has been widely adopted. In this paper, we addressed two pressing questions related to ALE meta-analysis: i) Which thresholding method is most appropriate to perform statistical inference? ii) Which sample size, i.e., number of experiments, is needed to perform robust meta-analyses? We provided quantitative answers to these questions by simulating more than 120,000 meta-analysis datasets using empirical parameters (i.e., number of subjects, number of reported foci, distribution of activation foci) derived from the BrainMap database. This allowed to characterize the behavior of ALE analyses, to derive first power estimates for neuroimaging meta-analyses, and to thus formulate recommendations for future ALE studies. We could show as a first consequence that cluster-level family-wise error (FWE) correction represents the most appropriate method for statistical inference, while voxel-level FWE correction is valid but more conservative. In contrast, uncorrected inference and false-discovery rate correction should be avoided. As a second consequence, researchers should aim to include at least 20 experiments into an ALE meta-analysis to achieve sufficient power for moderate effects. We would like to note, though, that these calculations and recommendations are specific to ALE and may not be extrapolated to other approaches for (neuroimaging) meta-analysis