Examining the reproducibility of meta-analyses in psychology:A preliminary report

Meta-analyses are an important tool to evaluate the literature. It is essential that meta-analyses can easily be reproduced to allow researchers to evaluate the impact of subjective choices on meta-analytic effect sizes, but also to update meta-analyses as new data comes in, or as novel statistical techniques (for example to correct for publication bias) are developed. Research in medicine has revealed meta-analyses often cannot be reproduced. In this project, we examined the reproducibility of meta-analyses in psychology by reproducing twenty published meta-analyses. Reproducing published meta-analyses was surprisingly difficult. 96% of meta-analyses published in 2013-2014 did not adhere to reporting guidelines. A third of these meta-analyses did not contain a table specifying all individual effect sizes. Five of the 20 randomly selected meta-analyses we attempted to reproduce could not be reproduced at all due to lack of access to raw data, no details about the effect sizes extracted from each study, or a lack of information about how effect sizes were coded. In the remaining meta-analyses, differences between the reported and reproduced effect size or sample size were common. We discuss a range of possible improvements, such as more clearly indicating which data were used to calculate an effect size, specifying all individual effect sizes, adding detailed information about equations that are used, and how multiple effect size estimates from the same study are combined, but also sharing raw data retrieved from original authors, or unpublished research reports. This project clearly illustrates there is a lot of room for improvement when it comes to the transparency and reproducibility of published meta-analyses

Close-to-threshold Meson Production in Hadronic Interactions

Studies of meson production at threshold in the hadron--hadron interaction began in the fifties when sufficient energies of accelerated protons were available. A strong interdependence between developments in accelerator physics, detector performance and theoretical understanding led to a unique vivid field of physics. Early experiments performed with bubble chambers revealed already typical ingredients of threshold studies, which were superseded by more complete meson production investigations at the nucleon beam facilities TRIUMF, LAMPF, PSI, LEAR and SATURNE. Currently, with the advent of the new cooler rings as IUCF, CELSIUS and COSY the field is entering a new domain of precision and the next step of further progress. The analysis of this new data in the short range limit permits a more fundamental consideration and a quantitative comparison of the production processes for different mesons in the few--body final states. The interpretation of the data take advantage of the fact that production reactions close-to-threshold are characterized by only a few degrees of freedom between a well defined combination of initial and exit channels. Deviations from predictions of phase-space controlled one-meson-exchange models are indications of new and exciting physics. Precision data on differential cross sections, isospin and spin observables -- partly but by no means adequately available -- are presently turning up on the horizon. There is work for the next years and excitement of the physics expected. Here we try to give a brief and at the same time comprehensive overview of this field of hadronic threshold production studies.Comment: 100 pages, Review article to be published in Prog. Part. Nucl. Phys. Vol. 49, issue 1 (2002

Examining the reproducibility of meta-analyses in psychology: A preliminary report

Meta-analyses are an important tool to evaluate the literature. It is essential that meta-analyses can easily be reproduced to allow researchers to evaluate the impact of subjective choices on meta-analytic effect sizes, but also to update meta-analyses as new data comes in, or as novel statistical techniques (for example to correct for publication bias) are developed. Research in medicine has revealed meta-analyses often cannot be reproduced. In this project, we examined the reproducibility of meta-analyses in psychology by reproducing twenty published meta-analyses. Reproducing published meta-analyses was surprisingly difficult. 96% of meta-analyses published in 2013-2014 did not adhere to reporting guidelines. A third of these meta-analyses did not contain a table specifying all individual effect sizes. Five of the 20 randomly selected meta-analyses we attempted to reproduce could not be reproduced at all due to lack of access to raw data, no details about the effect sizes extracted from each study, or a lack of information about how effect sizes were coded. In the remaining meta-analyses, differences between the reported and reproduced effect size or sample size were common. We discuss a range of possible improvements, such as more clearly indicating which data were used to calculate an effect size, specifying all individual effect sizes, adding detailed information about equations that are used, and how multiple effect size estimates from the same study are combined, but also sharing raw data retrieved from original authors, or unpublished research reports. This project clearly illustrates there is a lot of room for improvement when it comes to the transparency and reproducibility of published meta-analyses

Temporal and Spectral Analyses Reveal Individual Variation in a Non-Vocal Acoustic Display: The Drumming Display of the Ruffed Grouse (Bonasa umbellus, L.)

International audienc