Methods for estimating between-study variance and overall effect in meta-analysis of odds-ratios

In random-effects meta-analysis the between-study variance (τ 2) has a key role in assessing heterogeneity of study-level estimates and combining them to estimate an overall effect. For odds ratios the most common methods suffer from bias in estimating τ 2 and the overall effect and produce confidence intervals with below-nominal coverage. An improved approximation to the moments of Cochran's Q statistic, suggested by Kulinskaya and Dollinger (KD), yields new point and interval estimators of τ 2 and of the overall log-odds-ratio. Another, simpler approach (SSW) uses weights based only on study-level sample sizes to estimate the overall effect. In extensive simulations we compare our proposed estimators with established point and interval estimators for τ 2 and point and interval estimators for the overall log-odds-ratio (including the Hartung-Knapp-Sidik-Jonkman interval). Additional simulations included three estimators based on generalized linear mixed models and the Mantel-Haenszel fixed-effect estimator. Results of our simulations show that no single point estimator of τ 2 can be recommended exclusively, but Mandel-Paule and KD provide better choices for small and large numbers of studies, respectively. The KD estimator provides reliable coverage of τ 2. Inverse-variance-weighted estimators of the overall effect are substantially biased, as are the Mantel-Haenszel odds ratio and the estimators from the generalized linear mixed models. The SSW estimator of the overall effect and a related confidence interval provide reliable point and interval estimation of the overall log-odds-ratio

Introducing EMMIE: An evidence rating scale to encourage mixed-method crime prevention synthesis reviews

Objectives This short report describes the need for, and the development of, a coding system to distil the quality and coverage of systematic reviews of the evidence relating to crime prevention interventions. The starting point for the coding system concerns the evidence needs of policymakers and practitioners. Methods The coding scheme (EMMIE) proposed builds on previous scales that have been developed to assess the probity, coverage and utility of evidence both in health and criminal justice. It also draws on the principles of realist synthesis and review. Results The proposed EMMIE scale identifies five dimensions to which systematic reviews intended to inform crime prevention should speak. These are the Effect of intervention, the identification of the causal Mechanism(s) through which interventions are intended to work, the factors that Moderate their impact, the articulation of practical Implementation issues, and the Economic costs of intervention

From E-MAPs to module maps: dissecting quantitative genetic interactions using physical interactions

Recent technological breakthroughs allow the quantification of hundreds of thousands of genetic interactions (GIs) in Saccharomyces cerevisiae. The interpretation of these data is often difficult, but it can be improved by the joint analysis of GIs along with complementary data types. Here, we describe a novel methodology that integrates genetic and physical interaction data. We use our method to identify a collection of functional modules related to chromosomal biology and to investigate the relations among them. We show how the resulting map of modules provides clues for the elucidation of function both at the level of individual genes and at the level of functional modules

Salvage logging effects on regulating ecosystem services and fuel loads

We thank several authors who generously provided data for this meta-analysis (WebPanel 2). ABL acknowledges the support of mobility grants from Universidad de Alcalá and Spanish Ministry of Education, postdoctoral fellowships from the Spanish Ministry of Economy and the Alexander von Humboldt Foundation, and Project AS2013/MAE-2719 “REMEDINAL-3” from the Government of Madrid. The data underlying this paper are available through an institutional repository (http://hdl.handle.net/10481/62260).Salvage logging, or logging after natural disturbances such as wildfires, insect outbreaks, and windstorms, is carried out to recover some of a forest’s natural and/or economic capital. However, trade-offs between management objectives and a lack of consensus on the ecological consequences of salvage logging impair science-based decision making on the management of forests after natural disturbances. We conducted a global meta-analysis of the impacts of salvage logging on regulating ecosystem services and on fuel loads, as a frequent post-disturbance objective is preventing subsequent wildfires that could be fueled by the accumulation of dead trunks and branches. Salvage logging affected ecosystem services in a moderately negative way, regardless of disturbance type and severity, time elapsed since salvage logging, intensity of salvage logging, and the group of regulating ecosystem services being considered. However, prolonging the time between natural disturbance and salvage logging mitigated negative effects on regulating ecosystem services. Salvage logging had no overall effect on surface fuels; rather, different fuel types responded differently depending on the time elapsed since salvage logging. Delaying salvage logging by ~2–4 years may reduce negative ecological impacts without affecting surface fuel loads.Project AS2013/MAE-2719 “REMEDINAL-3” from the Government of Madri

Bonus versus penalty: how robust are the effects of contract framing?

We study the relative effectiveness of contracts that are framed either in terms of bonuses or penalties. In one set of treatments subjects know at the time of effort provision whether they have achieved the bonus / avoided the penalty. In another set of treatments subjects only learn the success of their performance at the end of the task. We fail to observe a contract framing effect in either condition: effort provision is statistically indistinguishable under bonus and penalty contracts

Methodological criteria for the assessment of moderators in systematic reviews of randomised controlled trials : a consensus study

Background: Current methodological guidelines provide advice about the assessment of sub-group analysis within RCTs, but do not specify explicit criteria for assessment. Our objective was to provide researchers with a set of criteria that will facilitate the grading of evidence for moderators, in systematic reviews. Method: We developed a set of criteria from methodological manuscripts (n = 18) using snowballing technique, and electronic database searches. Criteria were reviewed by an international Delphi panel (n = 21), comprising authors who have published methodological papers in this area, and researchers who have been active in the study of sub-group analysis in RCTs. We used the Research ANd Development/University of California Los Angeles appropriateness method to assess consensus on the quantitative data. Free responses were coded for consensus and disagreement. In a subsequent round additional criteria were extracted from the Cochrane Reviewers’ Handbook, and the process was repeated. Results: The recommendations are that meta-analysts report both confirmatory and exploratory findings for subgroups analysis. Confirmatory findings must only come from studies in which a specific theory/evidence based apriori statement is made. Exploratory findings may be used to inform future/subsequent trials. However, for inclusion in the meta-analysis of moderators, the following additional criteria should be applied to each study: Baseline factors should be measured prior to randomisation, measurement of baseline factors should be of adequate reliability and validity, and a specific test of the interaction between baseline factors and interventions must be presented. Conclusions: There is consensus from a group of 21 international experts that methodological criteria to assess moderators within systematic reviews of RCTs is both timely and necessary. The consensus from the experts resulted in five criteria divided into two groups when synthesising evidence: confirmatory findings to support hypotheses about moderators and exploratory findings to inform future research. These recommendations are discussed in reference to previous recommendations for evaluating and reporting moderator studies

Non-cognate translation priming in masked priming lexical decision experiments: a meta-analysis

The masked translation priming paradigm has been widely used in the last 25 years to investigate word processing in bilinguals. Motivated by studies reporting mixed findings, in particular for second language (L2) to first language (L1) translation priming, we conducted, for the first time in the literature, a meta-analysis of 64 lexical decision experiments across 24 studies to assess the effect sizes of L1-L2 and L2-L1 non-cognate translation priming effects in bilinguals. Our meta-analysis also investigated the influence of potential moderators of translation priming effects. The results provided clear evidence of significant translation priming effects for both directions, with L1-L2 translation priming significantly larger than L2-L1 translation priming (i.e., effect size of 0.86 vs. 0.31). The analyses also revealed that L1-L2 translation effect sizes were moderated by the interval between prime and target (ISI), whereas L2-L1 translation effect sizes were modulated by the number of items per cell. Theoretical and methodological implications of this meta-analysis are discussed and recommendations for future studies are provided

Sex differences in mathematics and reading achievement are inversely related: within- and across-nation assessment of 10 years of PISA data

We analyzed one decade of data collected by the Programme for International Student Assessment (PISA), including the mathematics and reading performance of nearly 1.5 million 15 year olds in 75 countries. Across nations, boys scored higher than girls in mathematics, but lower than girls in reading. The sex difference in reading was three times as large as in mathematics. There was considerable variation in the extent of the sex differences between nations. There are countries without a sex difference in mathematics performance, and in some countries girls scored higher than boys. Boys scored lower in reading in all nations in all four PISA assessments (2000, 2003, 2006, 2009). Contrary to several previous studies, we found no evidence that the sex differences were related to nations’ gender equality indicators. Further, paradoxically, sex differences in mathematics were consistently and strongly inversely correlated with sex differences in reading: Countries with a smaller sex difference in mathematics had a larger sex difference in reading and vice versa. We demonstrate that this was not merely a between-nation, but also a within-nation effect. This effect is related to relative changes in these sex differences across the performance continuum: We did not find a sex difference in mathematics among the lowest performing students, but this is where the sex difference in reading was largest. In contrast, the sex difference in mathematics was largest among the higher performing students, and this is where the sex difference in reading was smallest. The implication is that if policy makers decide that changes in these sex differences are desired, different approaches will be needed to achieve this for reading and mathematics. Interventions that focus on high-achieving girls in mathematics and on low achieving boys in reading are likely to yield the strongest educational benefits