The Synthesis of Regression Slopes in Meta-Analysis

Research on methods of meta-analysis (the synthesis of related study results) has dealt with many simple study indices, but less attention has been paid to the issue of summarizing regression slopes. In part this is because of the many complications that arise when real sets of regression models are accumulated. We outline the complexities involved in synthesizing slopes, describe existing methods of analysis and present a multivariate generalized least squares approach to the synthesis of regression slopes.Comment: Published in at http://dx.doi.org/10.1214/07-STS243 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Quasi-experimental study designs series –Paper 9: Collecting Data from Quasi-Experimental Studies

Objective: To identify variables that must be coded when synthesizing primary studies that use quasi-experimental designs.  Study Design and Setting: All quasi-experimental (QE) designs.  Results: When designing a systematic review of QE studies potential sources of heterogeneity – both theory-based and methodological – must be identified. We outline key components of inclusion criteria for syntheses of quasi-experimental studies. We provide recommendations for coding content-relevant and methodological variables, and outlined the distinction between bivariate effect sizes and partial (i.e., adjusted) effect sizes. Designs used and controls employed are viewed as of greatest importance. Potential sources of bias and confounding are also addressed.  Conclusion: Careful consideration must be given to inclusion criteria and the coding of theoretical and methodological variables during the design phase of a synthesis of quasi-experimental studies. The success of the meta-regression analysis relies on the data available to the meta-analyst. Omission of critical moderator variables (i.e., effect modifiers) will undermine the conclusions of a meta-analysis

Quasi-experimental study designs series – Paper 10: Synthesizing evidence for effects collected from quasi-experimental studies presents surmountable challenges

Objective: To outline issues of importance to analytic approaches to the synthesis of quasi-experiments (QEs), and to provide a statistical model for use in analysis. Study Design and Setting: We drew on the literatures of statistics, epidemiology, and social-science methodology to outline methods for synthesis of QE studies. The design and conduct of quasi-experiments, effect sizes from QEs, and moderator variables for the analysis of those effect sizes were discussed. Results: Biases, confounding, design complexities and comparisons across designs offer serious challenges to syntheses of QEs. Key components of meta-analyses of QEs were identified, including the aspects of QE study design to be coded and analyzed. Of utmost importance are the design and statistical controls implemented in the QEs. Such controls and any potential sources of bias and confounding must be modeled in analyses, along with aspects of the interventions and populations studied. Because of such controls, effect sizes from QEs are more complex than those from randomized experiments. A statistical meta-regression model that incorporates important features of the QEs under review was presented. Conclusion: Meta-analyses of quasi-experiments provide particular challenges, but thorough coding of intervention characteristics and study methods, along with careful analysis, should allow for sound inferences

Quiet Eye and Performance in Sport: A Meta-Analysis

Research linking the “quiet eye” (QE) period to subsequent performance has not been systematically synthesized. In this paper we review the literature on the link between the two through nonintervention (Synthesis 1) and intervention (Synthesis 2) studies. In the first synthesis, 27 studies with 38 effect sizes resulted in a large mean effect (d = 1.04) reflecting differences between experts’ and novices’ QE periods, and a moderate effect size (d = 0.58) comparing QE periods for successful and unsuccessful performances within individuals. Studies reporting QE duration as a percentage of the total time revealed a larger mean effect size than studies reporting an absolute duration (in milliseconds). The second synthesis of 9 articles revealed very large effect sizes for both the quiet-eye period (d = 1.53) and performance (d = 0.84). QE also showed some ability to predict performance effects across studies

Quasi-experimental study designs series-paper 6: risk of bias assessment.

OBJECTIVES: Rigorous and transparent bias assessment is a core component of high-quality systematic reviews. We assess modifications to existing risk of bias approaches to incorporate rigorous quasi-experimental approaches with selection on unobservables. These are nonrandomized studies using design-based approaches to control for unobservable sources of confounding such as difference studies, instrumental variables, interrupted time series, natural experiments, and regression-discontinuity designs. STUDY DESIGN AND SETTING: We review existing risk of bias tools. Drawing on these tools, we present domains of bias and suggest directions for evaluation questions. RESULTS: The review suggests that existing risk of bias tools provide, to different degrees, incomplete transparent criteria to assess the validity of these designs. The paper then presents an approach to evaluating the internal validity of quasi-experiments with selection on unobservables. CONCLUSION: We conclude that tools for nonrandomized studies of interventions need to be further developed to incorporate evaluation questions for quasi-experiments with selection on unobservables

Effects of Physical Activity on psychological Change in Advanced Age: A Multivariate Meta-Analysis

An example of multivariate meta-analysis is demonstrated by synthesizing the treatment effects of exercise of 15 groups on six mood state changes in elders measured by the Profile of Mood States (POMS) scale. Two different methods were used to analyze this multivariate dataset. The SAS codes for two set of the analyses were provided. Results showed that exercise has a modest and positive impact on elders mood change

Incorporating Quality Scores in Meta-Analysis

This paper examines the impact of quality-score weights in meta-analysis. A simulation examines the roles of study characteristics such as population effect size (ES) and its variance on the bias and mean square errors (MSEs) of the estimators for several patterns of relationship between quality and ES, and for specific patterns of systematic deviations related to quality differences. The bias and MSEs of the estimators are large when ESs from low-quality studies deviate from the population ES in specific ways, and bias does not approach zero in these cases. Because meta-analysts can never know whether biases due to quality exist, and because quality weights lead to bias in almost every condition studied, we recommend against the use of quality weights