Mapping reviews, scoping reviews, and evidence and gap maps (EGMs): the same but different— the “Big Picture” review family

Scoping reviews, mapping reviews, and evidence and gap maps are evidence synthesis methodologies that address broad research questions, aiming to describe a bigger picture rather than address a specific question about intervention effectiveness. They are being increasingly used to support a range of purposes including guiding research priorities and decision making. There is however a confusing array of terminology used to describe these different approaches. In this commentary, we aim to describe where there are differences in terminology and where this equates to differences in meaning. We demonstrate the different theoretical routes that underpin these differences. We suggest ways in which the approaches of scoping and mapping reviews may differ in order to guide consistency in reporting and method. We propose that mapping and scoping reviews and evidence and gap maps have similarities that unite them as a group but also have unique differences. Understanding these similarities and differences is important for informing the development of methods used to undertake and report these types of evidence synthesis

Current methods for development of rapid reviews about diagnostic tests: an international survey

Background Rapid reviews (RRs) have emerged as an efficient alternative to time-consuming systematic reviews—they can help meet the demand for accelerated evidence synthesis to inform decision-making in healthcare. The synthesis of diagnostic evidence has important methodological challenges. Here, we performed an international survey to identify the current practice of producing RRs for diagnostic tests. Methods We developed and administered an online survey inviting institutions that perform RRs of diagnostic tests from all over the world. Results All participants (N = 25) reported the implementation of one or more methods to define the scope of the RR; however, only one strategy (defining a structured question) was used by ≥90% of participants. All participants used at least one methodological shortcut including the use of a previous review as a starting point (92%) and the use of limits on the search (96%). Parallelization and automation of review tasks were not extensively used (48 and 20%, respectively). Conclusion Our survey indicates a greater use of shortcuts and limits for conducting diagnostic test RRs versus the results of a recent scoping review analyzing published RRs. Several shortcuts are used without knowing how their implementation affects the results of the evidence synthesis in the setting of diagnostic test reviews. Thus, a structured evaluation of the challenges and implications of the adoption of these RR methods is warranted

Challenges of rapid reviews for diagnostic test accuracy questions: a protocol for an international survey and expert consultation

Background: Assessment of diagnostic tests, broadly defined as any element that aids in the collection of additional information for further clarification of a patient’s health status, has increasingly become a critical issue in health policy and decision-making. Diagnostic evidence, including the accuracy of a medical test for a target condition, is commonly appraised using standard systematic review methodology. Owing to the considerable time and resources required to conduct these, rapid reviews have emerged as a pragmatic alternative by tailoring methods according to the decision maker’s circumstances. However, it is not known if streamlining methodological aspects has an impact on the validity of evidence synthesis. Furthermore, due to the particular nature and complexity of the appraisal of diagnostic accuracy, there is need for detailed guidance on how to conduct rapid reviews of diagnostic tests. In this study, we aim to identify the methods currently used by rapid review developers to synthesize evidence on diagnostic test accuracy, as well as to analyze potential shortcomings and challenges related to these methods. Methods: We will carry out a two-fold approach: (1) an international survey of professionals working in organizations that develop rapid reviews of diagnostic tests, in terms of the methods and resources used by these agencies when conducting rapid reviews, and (2) semi-structured interviews with senior-level individuals to further explore and validate the findings from the survey and to identify challenges in conducting rapid reviews. We will use STATA 15.0 for quantitative analyses and framework analysis for qualitative analyses. We will ensure protection of data during all stages. Discussion: The main result of this research will be a map of methods and resources currently used for conducting rapid reviews of diagnostic test accuracy, as well as methodological shortcomings and potential solutions in diagnostic knowledge synthesis that require further research

The Pandora's Box of Evidence Synthesis and the case for a living Evidence Synthesis Taxonomy.

Have we, as an evidence-based health community, opened the Pandora's box of evidence synthesis? There now exists a plethora of overlapping evidence synthesis approaches and duplicate, redundant and poor-quality reviews.1-4 After years of advocating for the need for systematic reviews of the evidence, there is a risk that this message been disseminated too widely and has been misinterpreted in this process. We have reached a point where in some fields more reviews exist than clinical trials, where same topic reviews are being conducted in parallel, and evidence syntheses possess limited utility for decision-making because of their poor quality or poor reporting.To paraphrase the late Douglas Altman,5 it is possible we are now at a stage where we need less reviews, better reviews and reviews done for the right reason - as opposed to the current state of mass production (approximately 80 reviews per day)6.Zachary Munn, Danielle Pollock, Timothy Hugh Barker, Jennifer Stone, Cindy Stern, Edoardo Aromataris, Holger J Schünemann, Barbara Clyne, Hanan Khalil, Reem A Mustafa, Christina Godfrey, Andrew Booth, Andrea C Tricco, Alan Pearso

Priority III: top 10 rapid review methodology research priorities identified using a James Lind Alliance Priority Setting Partnership

Objectives A rapid review is a form of evidence synthesis considered a resource-efficient alternative to the conventional systematic review. Despite a dramatic rise in the number of rapid reviews commissioned and conducted in response to the coronavirus disease 2019 pandemic, published evidence on the optimal methods of planning, doing, and sharing the results of these reviews is lacking. The Priority III study aimed to identify the top 10 unanswered questions on rapid review methodology to be addressed by future research. Study Design and Setting A modified James Lind Alliance Priority Setting Partnership approach was adopted. This approach used two online surveys and a virtual prioritization workshop with patients and the public, reviewers, researchers, clinicians, policymakers, and funders to identify and prioritize unanswered questions. Results Patients and the public, researchers, reviewers, clinicians, policymakers, and funders identified and prioritized the top 10 unanswered research questions about rapid review methodology. Priorities were identified throughout the entire review process, from stakeholder involvement and formulating the question, to the methods of a systematic review that are appropriate to use, through to the dissemination of results. Conclusion The results of the Priority III study will inform the future research agenda on rapid review methodology. We hope this will enhance the quality of evidence produced by rapid reviews, which will ultimately inform decision-making in the context of healthcare

What are the most important unanswered research questions on rapid review methodology? A James Lind Alliance research methodology Priority Setting Partnership: the Priority III study protocol [version 2; peer review: 2 approved, 2 approved with reservations]

Background: The value of rapid reviews in informing health care decisions is more evident since the onset of the coronavirus disease 2019 (COVID-19) pandemic. While systematic reviews can be completed rapidly, rapid reviews are usually a type of evidence synthesis in which components of the systematic review process may be simplified or omitted to produce information more efficiently within constraints of time, expertise, funding or any combination thereof. There is an absence of high-quality evidence underpinning some decisions about how we plan, do and share rapid reviews. We will conduct a modified James Lind Alliance Priority Setting Partnership to determine the top 10 unanswered research questions about how we plan, do and share rapid reviews in collaboration with patients, public, reviewers, researchers, clinicians, policymakers and funders. Methods: An international steering group consisting of key stakeholder perspectives (patients, the public, reviewers, researchers, clinicians, policymakers and funders) will facilitate broad reach, recruitment and participation across stakeholder groups. An initial online survey will identify stakeholders’ perceptions of research uncertainties about how we plan, do and share rapid reviews. Responses will be categorised to generate a long list of questions. The list will be checked against systematic reviews published within the past three years to identify if the question is unanswered. A second online stakeholder survey will rank the long list in order of priority. Finally, a virtual consensus workshop of key stakeholders will agree on the top 10 unanswered questions. Discussion: Research prioritisation is an important means for minimising research waste and ensuring that research resources are targeted towards answering the most important questions. Identifying the top 10 rapid review methodology research priorities will help target research to improve how we plan, do and share rapid reviews and ultimately enhance the use of high-quality synthesised evidence to inform health care policy and practice

Scoping reviews: time for clarity in definition, methods, and reporting

Objectives: The scoping review has become increasingly popular as a form of knowledge synthesis. However, a lack of consensus on scoping review terminology, definition, methodology, and reporting limits the potential of this form of synthesis. In this article, we propose recommendations to further advance the field of scoping review methodology. Study Design and Setting: We summarize current understanding of scoping review publication rates, terms, definitions, and methods. We propose three recommendations for clarity in term, definition and methodology. Results: We recommend adopting the terms “scoping review” or “scoping study” and the use of a proposed definition. Until such time as further guidance is developed, we recommend the use of the methodological steps outlined in the Arksey and O'Malley framework and further enhanced by Levac et al. The development of reportingguidance for the conduct and reporting of scoping reviews is underway. Conclusion: Consistency in the proposed domains and methodologies of scoping reviews, along with the development of reporting guidance, will facilitate methodological advancement, reduce confusion, facilitate collaboration and improve knowledge translation of scoping review findings

The role of scoping reviews in reducing research waste.

Objective: Scoping reviews and evidence map methodologies are increasingly being used by researchers. The objective of this article is to examine how scoping reviews can reduce research waste. Study design and setting: This article summarises the key issues facing the research community regarding research waste and how scoping reviews can make an important contribution to the reduction of research waste in both primary and secondary research. Results: The problem of research waste is an enduring challenge for global health, leading to a waste of human and financial resources and producing research outputs that do not provide answers to the most pressing research questions. Research waste occurs within primary research but also in secondary research such as evidence syntheses. The focus of scoping reviews on characterising the nature of existing evidence on a topic and of including all types of evidence, potentially reduces research waste in five ways: (i) identifying key research gaps on a topic, (ii) determining appropriate outcome measures, (iii) mapping existing methodological approaches, (iv) developing a consistent understanding of terms and concepts used in existing evidence, and (v) ensuring scoping reviews do not exacerbate the issue of research waste. Conclusion: To ensure that scoping reviews do not themselves end up contributing to research waste, it is important to register the scoping review and to ensure that international reporting standards and methodological guidance are followed