\ua9 2024 The Authors. Campbell Systematic Reviews published by John Wiley &amp; Sons Ltd on behalf of The Campbell Collaboration.Evidence and gap maps (EGMs) are an increasinly popular approach used in evidence synthesis. As an approach they address broad research questions, describing the existing evidence base, highlighting evidence gaps and providing an interactive visual tool for knowledge users. The purpose of this methodological study is to explore the the processes used in the development of EGM\u27s and how they are reported. The aim is to better understand current practice and identify where clearer guidance is needed to support their production

Bond M

Campbell F

Llewellyen JL

Pearson F

Stoniute A

Wong R

English

Newcastle University E-Prints

Received: 23 January 2023 | Accepted: 28 March 2024DOI: 10.1002/cl2.1402P RO TOCO L SMe t h o d sPROTOCOL: Methods used in the development, productionand updating of evidence and gap maps: A scoping reviewFiona Campbell1 | Ruth Wong2 | Jennifer L. Llewellyen2 | Akvile Stoniute1 |Fiona Pearson1 | Melissa Bond31Population Health Sciences Institute, Newcastle University, Newcastle, UK2School for Health and Related Research, University of Sheffield, Sheffield, UK3EPPI Centre, University College London, London, UKCorrespondenceFiona Campbell, Population Health SciencesInstitute, Newcastle University, Tyne andWear, Newcastle NE1 7RU, UK.Email: Fiona.campbell1@ncl.ac.ukFunding informationUK Prevention Research Partnership,Grant/Award Number: MR/S037578/2AbstractEvidence and gap maps (EGMs) are an increasinly popular approach used in evidencesynthesis. As an approach they address broad research questions, describing theexisting evidence base, highlighting evidence gaps and providing an interactive visualtool for knowledge users. The purpose of this methodological study is to explore thethe processes used in the development of EGM's and how they are reported. Theaim is to better understand current practice and identify where clearer guidance isneeded to support their production.K E YWORD Sevidence and gap maps, evidence synthesis, mapping reviews, scoping review, systematicreview1 | BACKGROUNDEvidence and Gap Maps (EGMs) sit within a family of evidencesynthesis methods that seek to address broader research ques-tions, such as; what interventions have been evaluated in thetreatment of COVID‐19 (Campbell et al., 2023), and what reviewlevel evidence has been published exploring elder abuse (Miktonet al., 2022). These types of broad research questions contrastwith the focused questions in an effectiveness systematic reviewwhere a PICO framework (population, intervention, comparator,and outcomes) helps to clearly specify the specific interventionthat is being evaluated, in which population, against whichcomparator and for what outcomes. Broader review questionsmay use an alternative question framework that allow for conceptsor multiple interventions or multiple study designs to beconsidered in one review. These types of evidence synthesisapproaches have gained greater prominence and have a particularvalue in supporting decision making where a wider understandingof an area is needed and gaps in knowledge need to be readilyidentified in order to direct future research priorities (Snilstveitet al., 2013). Within the context of the COVID‐19 pandemic,where rapid evidence was needed and new research was beingconducted and published within short time frames, evidencesynthesis approaches that allowed the range of evidence becomingavailable and freely accessible to a wide audience were needed.EGMs are a valuable tool in meeting these needs. They do notsynthesise existing evidence, but by locating, categorising, coding andpresenting the evidence in an interactive web‐based tool (Whiteet al., 2020), with links to the primary research, they offer a valuablevisualisation of existing evidence.Campbell Systematic Reviews. 2024;20:e1402. wileyonlinelibrary.com/journal/cl2 | 1 of 4https://doi.org/10.1002/cl2.1402This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium,provided the original work is properly cited.© 2024 The Authors. Campbell Systematic Reviews published by John Wiley & Sons Ltd on behalf of The Campbell Collaboration.A unique feature of these approaches is that the matrix intowhich the evidence is plotted, is generated a priori. Consultation withstakeholders and the use of theoretical frameworks will inform theconstruction of the dimensions within the framework. For example,when wanting to locate evidence for interventions to reduce elderabuse, all potential interventions are identified in the design of thematrix. The map can therefore show, not only the available evidence,but also the gaps where evidence is lacking.A barrier to the use of evidence in decision making is thatsystematic reviews are out of date or fail to be relevant to a particularcontext where a decision is being made. Advances in technologies andprocesses have created the foundations for frequent or ‘living’approaches to updating. During the COVID‐19 pandemic, clinicalpractice and policy decisions needed to be made quickly and beinformed by the explosion in COVID‐19 research. Several groups usedthe foundational work on living approaches in systematic reviews tolaunch COVID‐19 living systematic reviews, guidelines and EGMs (e.g.,Lorenc et al., 2023). For example, both theWHO and NICE, transitionedtheir COVID‐19 guidelines into living modes.Both living approaches and EGMS are still new and evolvingapproaches. There is little in the way of guidance and methods ofcreating a living map and approaches to updating maps. AlthoughCOVID‐19 was a catalyst to progressing some of these methods, thefrequency and implications of updates on policy and practice may varyfrom topic to topic. Recommendations for methods are likely totherefore need tailoring to the review topic (Elliott et al., 2014; Elliottet al., 2021).An EGM, with the use of filters to focus a search of existingevidence can therefore be particularly valuable. For example, theYouthEndowment Foundation EGM of interventions to prevention youngpeople getting involved in violence (https://youthendowmentfund.org.uk/wp-content/uploads/2022/04/Map-21-Apr-22-2.html) allows usersto locate evidence conducted in particular settings. Living reviews haveevolved as ways to address the problem of evidence becoming out ofdate and redundant. EGMs, as they do not synthesise the includedstudies, are one tool where the development of a living evidenceproduct faces fewer practical hurdles. The fact that these are also web‐based is a feature that means regular updates are practically easier.This review seeks to explore what methods of developing,maintaining and updating EGMs are currently used, and how.2 | REVIEW QUESTIONS(1) What methods are currently used in the development, produc-tion and updating of EGMs?(2) To what extent do EGMs adhere to recommended guidance(Campbell, PRISMA‐ScR)?(3) What procedures are in place for updating and maintain-ing EGMs?(4) What methodological guidance are being cited to support theapproaches used in the EGM process?3 | METHODSWe will use a scoping review methodology to address our broadresearch question (Campbell et al., 2023). It will draw on methodsdescribed by James et al. (2016), Arksey and O'Malley (2005) andLevac et al. (2010); including the location, screening, data extraction,coding and description of findings. The review will comply withPRISMA‐ScR (Tricco et al., 2018) reporting guidance.3.1 | Locating the EGMsInteractive visual representation of evidence gap maps can becreated by platform providers such as EPPI Mapper (Digital SolutionFoundry and EPPI‐Centre, 2022) and can be displayed on a public‐facing website. It is unclear whether all maps will have a relatedjournal publication describing the map and to our knowledge, there isno search filter on how to find evidence gap maps online. EGMscreated using EPPI Reviewer and EPPI Mapper will be systematicallylocated using the Google search engine. In addition, throughcollaboration with the EPPI Centre, we will undertake web‐searching for public‐facing maps that have been created using theEPPI Mapper tool. We will identify the number of unique maps (somemaps have several URLs) and search to obtain the supportingdocumentation describing the map such as associated publication,published report, technical guide and web resource. Other maps maybe identified by reference tracking of the supporting documentationof included maps.3.2 | Inclusion and exclusion criteria3.2.1 | Inclusion criteria• EGMs created by EPPI Mapper (Digital Solution Foundry andEPPI‐Centre, 2022)• Public‐facing online maps (with a URL)• Published in English3.2.2 | Exclusion criteria• EGMs created using other platforms such as the 3ie tool orEviMap.3.3 | Data extractionData concerning the presentation of the map, reporting ofmethods and methods for updating the map will be gathered bytwo reviewers working independently. Differences between thereviewer coding will be explored and discussed. Where necessary,2 of 4 | CAMPBELL ET AL. 18911803, 2024, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cl2.1402 by Newcastle University, Wiley Online Library on [13/05/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons Licensea third reviewer will be included in order to achieve consensus. Inparticular, the following information will be extracted, whereavailable:General(1) Website and host site for the map(2) Authors(3) Date of launch and map number (if applicable)(4) Who the intended end‐users are i.e. decision makers and/orresearchers(5) Location of accompanying protocol and report (links from map)(6) Team size, skills and roles(7) Software tools usedReview question and purpose(8) Research question/topic addressed by the EGM(9) Type of evidence included in the EGM(10) Date coverage of the evidence in the map e.g. from 2010 topresent(11) Description of methods used(a) Description of search strategy(b) Stakeholder engagement(c) Number of databases searched(d) Inclusion criteria defined(e) Types of studies included in the map(f) Methods of screening and data coding(g) Quality appraisal(h) Methods cited(12) Limitations of the evidence in the map e.g. language bias,exclusion of unpublished literature reportedUpdating and/or living maps(13) Any indication if/when the map will be updated(14) Description of ‘living’(15) Methods for updating(16) Use of automation in updating(a) Searching(b) Screening(c) Coding(d) Quality assessment(17) Funding for updating(18) Team roles in updating(19) Transparency of reporting when last search was undertakenAn area that will not be addressed in this mapping review, butnevertheless considered by the authors, are considerations of equity, andhow socially stratifying factors are accounted for in the map. Thisquestion is being addressed in a concurrent scoping review (Khalilet al., 2024).3.4 | Synthesis of findingsThe data extracted will be collated, tabulated and describednarratively.The framework for the description of the methods used willfollow the research questions and objectives of this review.• What methods are currently used in the development, productionand updating of EGMs?• To what extent to EGMs adhere to recommended guidance (Whiteet al., 2020; PRISMA‐ScR)?• What procedures are in place for updating and maintaining EGMs?• What methods are being cited by EGM authors guiding theresearch process?4 | CONCLUSIONThis scoping review will endeavour to describe the current state ofpractice in the development of evidence gap maps, a burgeoningapproach to evidence synthesis and one that has particular value inpolicy making (Snilstveit et al., 2013). It is an approach that is beingwidely adopted by organisations that fund research and seek toinform policy (e.g., 3ie, Youth Endowment Fund, Unicef). We want toexplore the extent to which these outputs are supported bypublished protocols, transparent and rigorous methodology, andhow efforts to maintain their relevance are ensured. The emergenceof ‘living maps’ will be included in our analysis and we will describehow ‘living’ is defined and operationalised in these reviews. Withtheir increasing popularity and their importance in shaping policy,ensuring these outputs are reliable and trustworthy sources ofevidence is important. A good understanding of current practice willprovide a sound basis for further development of existing reportingstandards such as PRISMA‐ScR which currently do not address thedevelopment of EGMs.We acknowledge that limiting our inclusion criteria to thoseEGMs produced using EPPI Mapper is a limitation of this review.Other mapping tools exist and are used within environmental science(https://www.oxsrev.org/systematic-mapping) and internationaldevelopment (https://www.3ieimpact.org/). However, the tools tosupport the development of these EGMs are created by specificreview groups with a particular topic focus, which limits theiradaptability to other topic areas, and they are not publicly available.EGMs created by EPPI Mapper can be adapted to any topic area andhosted on any website.CONTRIBUTIONS OF AUTHORSFC, FP and MB both have expertise in producing evidence and gapmaps and in using EPPI Reviewer and EPPI Mapper. FC conceived theproject, FC, RW and MB prepared the protocol. FC, AS and JL willCAMPBELL ET AL. | 3 of 4 18911803, 2024, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cl2.1402 by Newcastle University, Wiley Online Library on [13/05/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons Licenseundertake the data extraction. RW has expertise as an InformationSpecialist and will design the search strategies. All authors reviewedand agreed the protocol.DECLARATIONS OF INTERESTFC, RW, JL, AS and FP have no conflicts of interests to declare. MBworks at the EPPI Centre that created the EPPI Reviewer softwarebut does not receive any benefits personally from it, as UCL is a non‐profit organisation.PRELIMINARY TIMEFRAMEThe review will be submitted by September 2024.PLANS FOR UPDATING REVIEWFC will be responsible for updating the review. Funding to supportupdates beyond 2025 will be sought.SOURCES OF SUPPORT4.1 | Internal sourcesNone.4.2 | External sourcesThis work was supported by the UK Prevention Research Partnership(MR/S037578/2, Meier).REFERENCESArksey, H., & O'Malley, L. (2005). Scoping studies: Towards a methodologicalframework. International Journal of Social Research Methodology, 8(1),19–32.Campbell, F., Tricco, A. C., Munn, Z., Pollock, D., Saran, A., Sutton, A.,White, H., & Khalil, H. (2023). Mapping reviews, scoping reviews andevidence and gap maps (EGMs) – Same but different. The ‘BigPicture’ review family. Systematic Reviews, 12, 45.Digital Solution Foundry and EPPI‐Centre. (2022). EPPI‐mapper, version 2.1.0.EPPI‐Centre, UCL Social Research Institute, University College London.Elliott, J. H., Turner, T., Clavisi, O., Thomas, J., Higgins, J. P. T.,Mavergames, C., & Gruen, R. L. (2014). Living systematic reviews:An emerging opportunity to narrow the evidence‐practice gap. PLoSMedicine, 11(2), e1001603.Elliott, J., Lawrence, R., Minx, J. C., Oladapo, O. T., Ravaud, P.,Tendal Jeppesen, B., & Grimshaw, J. M. (2021). Decision makersneed constantly updated evidence synthesis. Nature Publishing Group.James, K. L., Randall, N. P., & Haddaway, N. R. (2016). A methodology forsystematic mapping in environmental sciences. EnvironmentalEvidence, 5(1), 7.Khalil, H. C. F., Daniel, K., Pollock, D., Munn, Z., Welsh, V., Saran, A.,Hoppe, D., & Tricco, A. C. (2024). Advancing the methodology ofmapping reviews: A scoping review. Research synthesis methods.Levac, D., Colquhoun, H., & O'Brien, K. K. (2010). Scoping studies:Advancing the methodology. Implementation Science, 5(1), 69.Lorenc, T., Khouja, C., Raine, G., Shemilt, I., Sutcliffe, K., D'Souza, P.,Burchett, H., Hinds, K., Macdowall, W., Melton, H., Richardson, M.,South, E., Stansfield, C., Thomas, S., Kwan, I., Wright, K., Sowden, A.,& Thomas, J. (2023). COVID‐19: Living map of the evidence. EPPI‐Centre, Social Science Research Unit, UCL Social Research Institute,University College London. https://eppi.ioe.ac.uk/cms/Projects/DepartmentofHealthandSocialCare/Publishedreviews(bydate)/COVID-19Livingsystematicmapoftheevidence/tabid/3765/Default.aspxMikton, C., Beaulieu, M., Yon, Y., Cadieux Genesse, J., St‐Martin, K.,Byrne, M., Phelan, A., Storey, J., Rogers, M., Campbell, F., & Ali, P.,(2022). PROTOCOL: Global elder abuse: A mega‐map of systematicreviews on prevalence, consequences, risk and protective factorsand interventions. Campbell systematic reviews, 18(2), p.e1227.Snilstveit, B., Vojtkova, M., Bhavsar, A., & Gaarder, M. (2013). Evidencegap maps‐a tool for promoting evidence‐informed policy andprioritizing future research. World bank policy research workingpaper (6725).Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D.,Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., Hempel, S.,Akl, E. A., Chang, C., McGowan, J., Stewart, L., Hartling, L.,Aldcroft, A., Wilson, M. G., Garritty, C., … Straus, S. E. (2018).PRISMA extension for scoping reviews (PRISMA‐ScR): Checklist andexplanation. Annals of Internal Medicine, 169(7), 467–473.White, H., Albers, B., Gaarder, M., Kornør, H., Littell, J., Marshall, Z.,Mathew, C., Pigott, T., Snilstveit, B., Waddington, H., & Welch, V.(2020). Guidance for producing a Campbell evidence and gap map.Campbell Systematic Reviews, 16(4), e1125.How to cite this article: Campbell, F., Wong, R., Llewellyen,J. L., Stoniute, A., Pearson, F., & Bond, M. (2024). PROTOCOL:Methods used in the development, production and updatingof evidence and gap maps: A scoping review. CampbellSystematic Reviews, 20, e1402.https://doi.org/10.1002/cl2.14024 of 4 | CAMPBELL ET AL. 18911803, 2024, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cl2.1402 by Newcastle University, Wiley Online Library on [13/05/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

PROTOCOL: Methods used in the development, production and updating of evidence and gap maps: A scoping review

https://eprints.ncl.ac.uk/file_store/production/298438/04A3BD95-6F91-4F0A-B781-EBC8EAE09505.pdf

PROTOCOL: Methods used in the development, production and updating of evidence and gap maps: A scoping review

Abstract

Similar works

Full text

Available Versions

Newcastle University E-Prints