What if we were Texas sharpshooters? Predictor reporting bias in regression analysis

The author analyzes reporting biases in regression analyses. The consequences of researchers’ strategy to select significant predictors and omit nonsignificant predictors from regression analyses are examined, focusing on how this strategy—labeled the Texas sharpshooter (TS) approach—creates a predictor reporting bias (PRB) in primary studies and research syntheses. PRB was demonstrated in simulation studies when correlation coefficients from several primary regression studies with an underlying TS approach were aggregated in meta-analyses. Several important findings are noted. First, meta-analytical effect sizes of true effects can be overestimated because smaller, nonsignificant findings are omitted from regression models. Second, suppression effects of correlated predictor variables create biased effect size estimations for variables that are not related to the outcome. Finally, existing small effects are concealed, and between-study heterogeneity can be overestimated. Results show that PRB is contingent on sample size. While PRB is substantial in studies with small sample sizes (N 500) are analyzed. Preconditions and remedies for reporting biases in regression analyses are discussed