Magnitude-Based Inference is Not Bayesian and is Not a Valid Method of Inference

journal articl

With great power comes great responsibility: Common errors in meta-analyses and meta-regressions in strength & conditioning research

Background and Objective: Meta-analysis and meta-regression are often highly cited and may influence practice. Unfortunately, statistical errors in meta-analyses are widespread and can lead to flawed conclusions. The purpose of this article was to review common statistical errors in meta-analyses and to document their frequency in highly cited meta-analyses from strength and conditioning research. Methods: We identified five errors in one highly cited meta-regression from strength and conditioning research: implausible outliers; overestimated effect sizes that arise from confusing standard deviation with standard error; failure to account for correlated observations; failure to account for within-study variance; and a focus on within-group rather than between-group results. We then quantified the frequency of these errors in 20 of the most highly cited meta-analyses in the field of strength and conditioning research from the past 20 years. Results: We found that 85 % of the 20 most highly cited meta-analyses in strength and conditioning research contained statistical errors. Almost half (45 %) contained at least one effect size that was mistakenly calculated using standard error rather than standard deviation. In several cases, this resulted in obviously wrong effect sizes, for example, effect sizes of 11 or 14 standard deviations. Additionally, 45 % failed to account for correlated observations despite including numerous effect sizes from the same study and often from the same group within the same study. Conclusions: Statistical errors in meta-analysis and meta-regression are common in strength and conditioning research. We highlight five errors that authors, editors, and readers should check for when preparing or critically reviewing meta-analyses

Raloxifene for women with Alzheimer disease: A randomized controlled pilot trial

OBJECTIVE: To determine whether raloxifene, a selective estrogen receptor modulator, improves cognitive function compared with placebo in women with Alzheimer disease (AD) and to provide an estimate of cognitive effect. METHODS: This pilot study was conducted as a randomized, double-blind, placebo-controlled trial, with a planned treatment of 12 months. Women with late-onset AD of mild to moderate severity were randomly allocated to high-dose (120 mg) oral raloxifene or identical placebo provided once daily. The primary outcome compared between treatment groups at 12 months was change in the Alzheimer's Disease Assessment Scale, cognitive subscale (ADAS-cog). RESULTS: Forty-two women randomized to raloxifene or placebo were included in intent-to-treat analyses (mean age 76 years, range 68-84), and 39 women contributed 12-month outcomes. ADAS-cog change scores at 12 months did not differ significantly between treatment groups (standardized difference 0.03, 95% confidence interval -0.39 to 0.44, 2-tailed p = 0.89). Raloxifene and placebo groups did not differ significantly on secondary analyses of dementia rating, activities of daily living, behavior, or a global cognition composite score. Caregiver burden and caregiver distress were similar in both groups. CONCLUSIONS: Results on the primary outcome showed no cognitive benefits in the raloxifene-treated group. CLASSIFICATION OF EVIDENCE: This study provides Class I evidence that for women with AD, raloxifene does not have a significant cognitive effect. The study lacked the precision to exclude a small effect