Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs.Fil: Christie, Alec P.. University of Cambridge; Reino UnidoFil: Abecasis, David. Universidad de Algarve. Centro de Ciencias del Mar; PortugalFil: Adjeroud, Mehdi. Université de Perpignan; Francia. Institut de Recherche Pour Le Developpement; FranciaFil: Alonso, Juan Carlos. Consejo Superior de Investigaciones Científicas. Museo Nacional de Ciencias Naturales; EspañaFil: Amano, Tatsuya. University of Queensland; AustraliaFil: Anton, Alvaro. Universidad del País Vasco. Facultad de Educación de Bilbao; EspañaFil: Baldigo, Barry P.. United States Geological Survey; Estados UnidosFil: Barrientos, Rafael. Universidad Complutense de Madrid; EspañaFil: Bicknell, Jake E.. University of Kent; Reino UnidoFil: Buhl, Deborah A.. United States Geological Survey; Estados UnidosFil: Cebrian, Just. Mississippi State University; Estados UnidosFil: Ceia, Ricardo S.. Universidad de Coimbra; PortugalFil: Cibils Martina, Luciana. Universidad Nacional de Río Cuarto. Facultad de Ciencias Exactas, Fisicoquímicas y Naturales. Departamento de Ciencias Naturales; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; ArgentinaFil: Clarke, Sarah. Marine Institute; IrlandaFil: Claudet, Joachim. Universite de Paris; Francia. Centre National de la Recherche Scientifique; FranciaFil: Craig, Michael D.. University of Western Australia; Australia. Murdoch University; AustraliaFil: Davoult, Dominique. Sorbonne University; FranciaFil: De Backer, Annelies. Flanders Research Institute for Agriculture, Fisheries and Food; BélgicaFil: Donovan, Mary K.. University of California; Estados Unidos. University of Hawaii at Manoa; Estados UnidosFil: Eddy, Tyler D.. University of South Carolina; Estados Unidos. Memorial University of Newfoundland; Canadá. Victoria University of Wellington; Nueva ZelandaFil: França, Filipe M.. Lancaster University; Reino UnidoFil: Gardner, Jonathan P. A.. Victoria University of Wellington; Nueva ZelandaFil: Harris, Bradley P.. Alaska Pacific University; Estados UnidosFil: Huusko, Ari. Natural Resources Institute Finland; FinlandiaFil: Jones, Ian L.. Memorial University of Newfoundland; CanadáFil: Kelaher, Brendan P.. Southern Cross University; AustraliaFil: Kotiaho, Janne S.. Universidad de Jyvaskyla; FinlandiaFil: López Baucells, Adrià. Universidad de Lisboa; Portugal. Smithsonian Tropical Research Institute; Panamá. Universidad Nacional de Colombia. Instituto de Investigaciones Amazonicas; Colombia. Museo de Ciencias Naturales de Granollers; EspañaFil: Major, Heather L.. University of New Brunswick; CanadáFil: Mäki Petäys, Aki. Voimalohi Oy; Finlandia. University of Oulu; Finlandi

Quantifying and addressing the prevalence and biasof study designs in the environmental and social sciences

Building trust in science and evidence-based decision-making depends heavily on the cred-ibility of studies and theirfindings. Researchers employ many different study designs thatvary in their risk of bias to evaluate the true effect of interventions or impacts. Here, weempirically quantify, on a large scale, the prevalence of different study designs and themagnitude of bias in their estimates. Randomised designs and controlled observationaldesigns with pre-intervention sampling were used by just 23% of intervention studies inbiodiversity conservation, and 36% of intervention studies in social science. We demonstrate,through pairwise within-study comparisons across 49 environmental datasets, that thesetypes of designs usually give less biased estimates than simpler observational designs. Wepropose a model-based approach to combine study estimates that may suffer from differentlevels of study design bias, discuss the implications for evidence synthesis, and how tofacilitate the use of more credible study designs.Peer reviewe

Probabilistic relations between acid-base chemistry and fish assemblages in streams of the western Adirondack Mountains, New York, USA

Surface waters across much of New Yorkâ s Adirondack Mountains were acidified in the late 20th century but began to recover following the 1990 amendments to the Clean Air Act. Little data, however, were available to characterize biological impacts and predict recovery of fish assemblages in streams of the region. Quantitative fish and chemistry surveys were completed in 47 headwater streams during summer 2014-16 to develop logistic (probabilistic) models that characterize the status of contemporary fish assemblages and predict how different N and S deposition loads may affect future fish assemblages. Models for Ali and richness â Ľ1 species; and for ANC and total density >400 fish/0.1ha, total biomass >1500 g/0.1ha, Brook Trout Salvelinus fontinalis density >0 or >200 fish/0.1ha, and Brook Trout biomass >1000 g/0.1ha were suitable for evaluating community and population responses to changes in acid-base chemistry. Anticipated changes in national (U.S.) secondary standards for atmospheric emissions of NOx and SOx to achieve target N and S deposition loads will alter acid-base chemistry and the probabilities for observing various levels of fish metrics in streams across the region and elsewhere.The accepted manuscript in pdf format is listed with the files at the bottom of this page. The presentation of the authors' names and (or) special characters in the title of the manuscript may differ slightly between what is listed on this page and what is listed in the pdf file of the accepted manuscript; that in the pdf file of the accepted manuscript is what was submitted by the author