The importance of meta-analysis and systematic review: How research legacy can be maximized through adequate reporting

Systematic reviews are widely accepted as a ‘gold standard’ in evidence synthesis and the meta-analysis within provides a powerful means of looking across datasets. Neal Haddaway argues that while certain fields have embraced these reviews, there is a great opportunity for their growth in other fields. One way to encourage secondary synthesis is for researchers to ensure their data is reported in sufficient detail. Thinking carefully about legacy and future use of data is not only sensible, but should be an obligation

8 common problems with literature reviews and how to fix them

Literature reviews are an integral part of the process and communication of scientific research. Whilst systematic reviews have become regarded as the highest standard of evidence synthesis, many literature reviews fall short of these standards and may end up presenting biased or incorrect conclusions. In this post, Neal Haddaway highlights 8 common problems with literature review methods, provides examples for each and provides practical solutions for ways to mitigate them

Aquatic macroinvertebrate responses to native and non-native predators

Non-native species can profoundly affect native ecosystems through trophic interactions with native species. Native prey may respond differently to non-native versus native predators since they lack prior experience. Here we investigate antipredator responses of two common freshwater macroinvertebrates, Gammarus pulex and Potamopyrgus jenkinsi, to olfactory cues from three predators; sympatric native fish (Gasterosteus aculeatus), sympatric native crayfish (Austropotamobius pallipes), and novel invasive crayfish (Pacifastacus leniusculus). G. pulex responded differently to fish and crayfish; showing enhanced locomotion in response to fish, but a preference for the dark over the light in response to the crayfish. P. jenkinsi showed increased vertical migration in response to all three predator cues relative to controls. These different responses to fish and crayfish are hypothesised to reflect the predators’ differing predation types; benthic for crayfish and pelagic for fish. However, we found no difference in response to native versus invasive crayfish, indicating that prey naiveté is unlikely to drive the impacts of invasive crayfish. The Predator Recognition Continuum Hypothesis proposes that benefits of generalisable predator recognition outweigh costs when predators are diverse. Generalised responses of prey as observed here will be adaptive in the presence of an invader, and may reduce novel predators’ potential impacts

Open Synthesis : on the need for evidence synthesis to embrace Open Science

Abstract: The Open Science movement can be broadly summarised as aiming to promote integrity, repeatability and transparency across all aspects of research, from data collection to publication. Systematic reviews and systematic maps aim to provide a reliable synthesis of the evidence on a particular topic, making use of methods that seek to maximise repeatability and comprehensives whilst minimising subjectivity and bias. The central tenet of repeatability is operationalised by transparently reporting methodological activities in detail, such that all actions could be replicated and verified. To date, evidence synthesis has only partially embraced Open Science, typically striving for Open Methodology and Open Access, and occasionally providing sufficient information to be considered to have Open Data for some published reviews. Evidence synthesis communities needs to better embrace Open Science not only to balance knowledge access and increase efficiency, but also to increase reliability, trust and reuse of information collected and synthesised within a review: concepts fundamental to systematic reviews and maps. All aspects of Open Science should be embraced: Open Methodology, Open Data, Open Source and Open Access. In doing so, evidence synthesis can be made more equal, more efficient and more trustworthy. I provide concrete recommendations of how CEE and others can fully embrace Open Synthesis

Planetary health as a laboratory for enhanced evidence synthesis

Abstract : Please refer to full text to view abstract

A methodology for systematic mapping in environmental sciences

Systematic mapping was developed in social sciences in response to a lack of empirical data when answering questions using systematic review methods, and a need for a method to describe the literature across a broad subject of interest. Systematic mapping does not attempt to answer a specific question as do systematic reviews, but instead collates, describes and catalogues available evidence (e.g. primary, secondary, theoretical, economic) relating to a topic or question of interest. The included studies can be used to identify evidence for policy-relevant questions, knowledge gaps (to help direct future primary research) and knowledge clusters (sub-sets of evidence that may be suitable for secondary research, for example systematic review). Evidence synthesis in environmental sciences faces similar challenges to those found in social sciences. Here we describe the translation of systematic mapping methodology from social sciences for use in environmental sciences. We provide the first process-based methodology for systematic maps, describing the stages involved: establishing the review team and engaging stakeholders; setting the scope and question; setting inclusion criteria for studies; scoping stage; protocol development and publication; searching for evidence; screening evidence; coding; production of a systematic map database; critical appraisal (optional); describing and visualising the findings; report production and supporting information. We discuss the similarities and differences in methodology between systematic review and systematic mapping and provide guidance for those choosing which type of synthesis is most suitable for their requirements. Furthermore, we discuss the merits and uses of systematic mapping and make recommendations for improving this evolving methodology in environmental sciences

Comparing smartphones to tablets for face-to-face interviewing in Kenya

Research conducted over the past 30 years has demonstrated a reduction in errors and improvement in data quality when face–to-face social surveys are carried out using computers instead of paper and pencil. However, research examining the quality of data collected by interviewers using mobile devices is in its infancy and is based in developed countries. In a small pilot study conducted during the World Bank’s Kenya State of the Cities Baseline Survey, a face-to-face survey on living conditions, infrastructure and service delivery, the authors compared the quality of data collected using smartphones to data collected using tablets. The study of mobile touchscreen devices showed that tablets outperformed phones in some cases, but that the results were highly dependent on the interviewer

Evidence for the effects of neonicotinoids used in arable crop production on non-target organisms and concentrations of residues in relevant matrices: a systematic map protocol

Background Neonicotinoid insecticides (NNIs) have been routinely used in arable crop protection since their development in the early 1990s. These insecticides have been subject to the same registration procedures as other groups of pesticides, thus meet the same environmental hazard standards as all crop protection products. However, during the last 10 years the debate regarding their possible detrimental impact on non-target organisms, particularly pollinators, has become increasingly contentious and widely debated. Against this background, legislators and politicians in some countries, have been faced with a need to make decisions on the future registration of some or all of this class of insecticides, based on published evidence that in some areas is incomplete or limited in extent. This has created much concern in agricultural communities that consider that the withdrawal of these insecticides is likely to have significant negative economic, socio-economic and environmental consequences. Methods The proposed systematic map aims to address the following primary question: What is the available evidence for the effects of neonicotinoids used in arable crop production on non-target organisms and concentrations of residues in relevant matrices? The primary question will be divided into two sub-questions to gather research literature for (1) the effect of NNIs on non-target organisms (2) the occurrence of concentrations of NNIs in matrices of relevance to non-target organisms (i.e. exposure routes). The systematic map will focus on NNIs used in arable crop production: imidacloprid, clothianidin, thiamethoxam, acetamiprid, thiacloprid and dinotefuran. Separate inclusion criteria have been developed for each sub-question. Traditional academic and grey literature will be searched for in English language and a searchable databases containing extracted meta-data from relevant included studies will be developed

Updating and amending systematic reviews and systematic maps in environmental management

Systematic reviews and systematic maps aim to provide an overview of the best available evidence to inform research, policy and practice. However, like any form of review, they will require updating periodically to ensure that the most recent evidence has been incorporated. Here we outline two types of review revisions as recognised in medicine: updates and amendments. Updates involve a search for new studies, expanding the evidence base through time. Any other change (e.g. in screening or synthesis) or correction to the original report is an amendment. Decisions as to whether/when it is appropriate to undertake an update or amendment must be made on a case-by-case basis, considering issues such as the reliability and scope of the existing review or map, likely volume of new evidence, resources available, and the likely value of including new information. Careful, consistent reporting is necessary to ensure transparency and repeatability, particularly where there are deviations from the original methods, and authors should highlight key advances relative to the original report. Updating environmental systematic reviews and maps will be an increasingly important activity as the numbers of both primary studies and synthetic reports in the literature continue to grow