The use of ‘PICO for synthesis’ and methods for synthesis without meta-analysis: protocol for a survey of current practice in systematic reviews of health interventions

INTRODUCTION: Systematic reviews involve synthesis of research to inform decision making by clinicians, consumers, policy makers and researchers. While guidance for synthesis often focuses on meta-analysis, synthesis begins with specifying the ’PICO for each synthesis’ (i.e. the criteria for deciding which populations, interventions, comparators and outcomes are eligible for each analysis). Synthesis may also involve the use of statistical methods other than meta-analysis (e.g. vote counting based on the direction of effect, presenting the range of effects, combining P values) augmented by visual display, tables and text-based summaries. This study examines these two aspects of synthesis. OBJECTIVES: To identify and describe current practice in systematic reviews of health interventions in relation to: (i) approaches to grouping and definition of PICO characteristics for synthesis; and (ii) methods of summary and synthesis when meta-analysis is not used. METHODS: We will randomly sample 100 systematic reviews of the quantitative effects of public health and health systems interventions published in 2018 and indexed in the Health Evidence and Health Systems Evidence databases. Two authors will independently screen citations for eligibility. Two authors will confirm eligibility based on full text, then extract data for 20% of reviews on the specification and use of PICO for synthesis, and the presentation and synthesis methods used (e.g. statistical synthesis methods, tabulation, visual displays, structured summary). The remaining reviews will be confirmed as eligible and data extracted by a single author. We will use descriptive statistics to summarise the specification of methods and their use in practice. We will compare how clearly the PICO for synthesis is specified in reviews that primarily use meta-analysis and those that do not. CONCLUSION: This study will provide an understanding of current practice in two important aspects of the synthesis process, enabling future research to test the feasibility and impact of different approaches

Critical elements of synthesis questions are incompletely reported: survey of systematic reviews of intervention effects

Objectives: To examine the characteristics of population, intervention and outcome groups and the extent to which they were completely reported for each synthesis in a sample of systematic reviews (SRs) of interventions. Study design and setting: We coded groups that were intended (or used) for comparisons in 100 randomly sampled SRs of public health and health systems interventions published in 2018 from the Health Evidence and Health Systems Evidence databases. Results: Authors commonly used population, intervention and outcome groups to structure comparisons, but these groups were often incompletely reported. For example, of 41 SRs that identified and/or used intervention groups for comparisons, 29 (71%) identified the groups in their methods description before reporting of the results (e.g., in the Background or Methods), 12 (29%) defined the groups in enough detail to replicate decisions about which included studies were eligible for each synthesis, 6 (15%) provided a rationale, and 24 (59%) stated that the groups would be used for comparisons. Sixteen (39%) SRs used intervention groups in their synthesis without any mention in the methods. Reporting for population, outcome and methodological groups was similarly incomplete. Conclusion: Complete reporting of the groups used for synthesis would improve transparency and replicability of reviews, and help ensure that the synthesis is not driven by what is reported in the included studies. Although concerted effort is needed to improve reporting, this should lead to more focused and useful reviews for decision-makers

Current practice in systematic reviews including the ‘PICO for each synthesis’ and methods other than meta-analysis: protocol for a cross-sectional study [version 1; peer review: awaiting peer review]

INTRODUCTION: Systematic reviews are used to synthesise research and inform decision making by clinicians, consumers and policy makers. The synthesis component of systematic reviews is often narrowly considered as the use of statistical methods to combine the results of studies, primarily meta-analysis. However, synthesis can be considered more broadly as a process beginning with: (i) defining the groupings of populations, interventions and outcomes to be compared (the ‘PICO for each synthesis’); (ii) examining the characteristics of the available studies; and (iii) applying synthesis methods from among multiple options. To date, there has been limited examination of approaches used in reviews to define and group PICO characteristics and synthesis methods other than meta-analysis. OBJECTIVES: To identify and describe current practice in systematic reviews in relation to structuring the PICO for each synthesis and methods for synthesis when meta-analysis is not used. METHODS: We will randomly sample 100 systematic reviews of the effects of public health and health systems interventions published in 2018 and indexed in the Health Evidence and Health Systems Evidence databases. Two authors will independently screen studies for eligibility. One author will extract data on approaches to grouping and defining populations, interventions and outcomes, and the rationale for the chosen groups; and the presentation and synthesis methods used (e.g. tabulation, visual displays, statistical synthesis methods such as combining P values, vote counting based on direction of effect). A second author will undertake independent data extraction for a subsample of reviews. Descriptive statistics will be used to summarise the findings. Specifically, we will compare approaches to grouping in reviews that primarily use meta-analysis versus those that do not. CONCLUSION: This study will provide an understanding of current practice in two important aspects of the synthesis process, enabling future research to test the feasibility and impact of different methodological approaches