The importance of meta-analysis and systematic review: How research legacy can be maximized through adequate reporting

Systematic reviews are widely accepted as a ‘gold standard’ in evidence synthesis and the meta-analysis within provides a powerful means of looking across datasets. Neal Haddaway argues that while certain fields have embraced these reviews, there is a great opportunity for their growth in other fields. One way to encourage secondary synthesis is for researchers to ensure their data is reported in sufficient detail. Thinking carefully about legacy and future use of data is not only sensible, but should be an obligation

8 common problems with literature reviews and how to fix them

Literature reviews are an integral part of the process and communication of scientific research. Whilst systematic reviews have become regarded as the highest standard of evidence synthesis, many literature reviews fall short of these standards and may end up presenting biased or incorrect conclusions. In this post, Neal Haddaway highlights 8 common problems with literature review methods, provides examples for each and provides practical solutions for ways to mitigate them

Open Synthesis : on the need for evidence synthesis to embrace Open Science

Abstract: The Open Science movement can be broadly summarised as aiming to promote integrity, repeatability and transparency across all aspects of research, from data collection to publication. Systematic reviews and systematic maps aim to provide a reliable synthesis of the evidence on a particular topic, making use of methods that seek to maximise repeatability and comprehensives whilst minimising subjectivity and bias. The central tenet of repeatability is operationalised by transparently reporting methodological activities in detail, such that all actions could be replicated and verified. To date, evidence synthesis has only partially embraced Open Science, typically striving for Open Methodology and Open Access, and occasionally providing sufficient information to be considered to have Open Data for some published reviews. Evidence synthesis communities needs to better embrace Open Science not only to balance knowledge access and increase efficiency, but also to increase reliability, trust and reuse of information collected and synthesised within a review: concepts fundamental to systematic reviews and maps. All aspects of Open Science should be embraced: Open Methodology, Open Data, Open Source and Open Access. In doing so, evidence synthesis can be made more equal, more efficient and more trustworthy. I provide concrete recommendations of how CEE and others can fully embrace Open Synthesis

Response to “Every ROSE has its thorns”

Abstract: Sharp et al. [1] raise a number of concerns about the development and communication of ROSES (RepOrting standards for Systematic Evidence Syntheses), and we welcome the opportunity to explain some of the underlying thinking behind development of the reporting standards for environmental evidence syntheses

ROSES RepOrting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps.

Reliable synthesis of the various rapidly expanding bodies of evidence is vital for the process of evidence-informed decision-making in environmental policy, practice and research. With the rise of evidence-base medicine and increasing numbers of published systematic reviews, criteria for assessing the quality of reporting have been developed. First QUOROM (Lancet 354:1896–1900, 1999) and then PRISMA (Ann Intern Med 151:264, 2009) were developed as reporting guidelines and standards to ensure medical meta-analyses and systematic reviews are reported to a high level of detail. PRISMA is now widely used by a range of journals as a pre-submission checklist. However, due to its development for systematic reviews in healthcare, PRISMA has limited applicability for reviews in conservation and environmental management. We highlight 12 key problems with the application of PRISMA to this field, including an overemphasis on meta-analysis and no consideration for other synthesis methods. We introduce ROSES (RepOrting standards for Systematic Evidence Syntheses), a pro forma and flow diagram designed specifically for systematic reviews and systematic maps in the field of conservation and environmental management. We describe how ROSES solves the problems with PRISMA. We outline the key benefits of our approach to designing ROSES, in particular the level of detail and inclusion of rich guidance statements. We also introduce the extraction of meta-data that describe key aspects of the conduct of the review. Collated together, this summary record can help to facilitate rapid review and appraisal of the conduct of a systematic review or map, potentially speeding up the peer-review process. We present the results of initial road testing of ROSES with systematic review experts, and propose a plan for future development of ROSES

Mapping the predicted and potential impacts of metal mining and its mitigation measures in Arctic and boreal regions using environmental and social impact assessments: a systematic map protocol

Abstract : Background: Since the 1960s, environmental impact assessments (EIAs) and recently, social impact assessments (SIAs), have been conducted during the planning stages of large development projects to identify potential adverse effects and propose mitigation measures to ameliorate these impacts. EIAs and SIAs should outline all possible posi- tive and negative effects of a proposed action or development on ecological and social systems, respectively, includ- ing biodiversity, flora and fauna, abiotic components (such as air quality), human health, security and wellbeing. The work outlined herein aims to generate a list of all possible direct and indirect effects of metal mining (including gold, iron, copper, nickel, zinc, silver, molybdenum and lead) along with the impacts of mitigation measures proposed, that are mentioned in EIAs and SIAs in Arctic and boreal regions of the following countries/regions: Canada, Alaska (USA), Greenland, the Faroe Islands, Iceland, Norway, Sweden, Finland and Russia. Methods: We will conduct searches for environmental and social impact assessments in Swedish and English, and until theoretical saturation is reached (i.e. no new action-impact pathways are identified). We will perform searches of specialist websites (e.g. public repositories of environmental and social impact assessments) and Google Scholar. We will also contact relevant stakeholders (that have been identified in the wider 3MK project https://osf.io/cvh3u/) and make a call for additional information. Eligibility screening will be conducted at two levels: title and full text. Meta-data will be extracted from eligible studies including type of mining activity, location of mine, type of impacts, and planned mitigation measures. Findings will be presented narratively, in a searchable relational database and in an evidence altas (a cartographic map). We will produce a framework of different mining impacts and related mitiga- tion measures from practitioners’ knowledge reflected in EIAs and SIAs. This framework will further form the basis of a multiple knowledge base on mining impacts and mitigation measures generated from different knowledges includ- ing scientific, Indigenous, and practitioners’ knowledge

What ecotechnologies exist for recycling carbon and nutrients from domestic wastewater? A systematic map protocol

Abstract: Background: Pollution of the Baltic Sea continues to be a problem. Major terrestrial sources of nutrient emissions to the Baltic Sea are agriculture and wastewater, both major causes of eutrophication. Wastewater contains nutrients and organic matter that could constitute valuable products such as agricultural fertilizers and source of energy. With the EU’s action plan for circular economy, waste management and resource utilization is central. Thus the integration of resource recovery to wastewater management could create benefits beyond the wastewater sector. There is a growing interest in resource recovery from wastewater. However, there is no systematic overview of the literature on technologies to recover nutrients and carbon from wastewater sources done to date. Methods: This systematic map will identify a representative list of studies on ecotechnologies for reusing carbon and nutrients (nitrogen and phosphorus) from domestic wastewater, which includes e.g. sewage sludge and wastewater fractions. Searches will be performed in five bibliographic databases, one search engine and 38 specialist websites. Searches will mainly be performed in English, search for literature in specialist websites will also include Finnish, Polish and Swedish. Coding and meta-data extraction will include information on ecotechnology name and short description, reuse outcome (i.e. reuse of carbon, nitrogen and/or phosphorus), type of reuse (i.e. whether it is explicit or implicit), study country and location, latitude and longitude. All screening and coding will be done after initial consistency checking. The outcomes of this systematic map will be a searchable database of coded studies. Findings will be presented in a geo-informational system (i.e. an evidence atlas) and knowledge gaps and clusters will be visualised via heat maps

Research Weaving: Visualizing the Future of Research Synthesis

We propose a new framework for research synthesis of both evidence and influence, named research weaving. It summarizes and visualizes information content, history, and networks among a collection of documents on any given topic. Research weaving achieves this feat by combining the power of two methods: systematic mapping and bibliometrics. Systematic mapping provides a snapshot of the current state of knowledge, identifying areas needing more research attention and those ready for full synthesis. Bibliometrics enables researchers to see how pieces of evidence are connected, revealing the structure and development of a field. We explain how researchers can use some or all of these tools to gain a deeper, more nuanced understanding of the scientific literature