The effectiveness of Payments for Ecosystem Services at delivering improvements in water quality: lessons for experiments at the landscape scale

Background Randomised Control Trials (RCTs) are used in impact evaluation in a range of fields. However, despite calls for their greater use in environmental management, their use to evaluate landscape scale interventions remains rare. Payments for Ecosystem Services (PES) incentivise land users to manage land to provide environmental benefits. We present the first RCT evaluation of a PES program aiming to improve water quality. Watershared is a program which incentivises landowners to avoid deforestation and exclude cattle from riparian forests. Using this unusual landscape-scale experiment we explore the efficacy of Watershared at improving water quality, and draw lessons for future RCT evaluations of landscape-scale environmental management interventions. Methods One hundred and twenty-nine communities in the Bolivian Andes were randomly allocated to treatment (offered Watershared agreements) or control (not offered agreements) following baseline data collection (including Escherichia coli contamination in most communities) in 2010. We collected end-line data in 2015. Using our end-line data, we explored the extent to which variables associated with the intervention (e.g. cattle exclusion, absence of faeces) predict water quality locally. We then investigated the efficacy of the intervention at improving water quality at the landscape scale using the RCT. This analysis was done in two ways; for the subset of communities for which we have both baseline and end-line data from identical locations we used difference-in-differences (matching on baseline water quality), for all sites we compared control and treatment at end-line controlling for selected predictors of water quality. Results The presence of cattle faeces in water adversely affected water quality suggesting excluding cattle has a positive impact on water quality locally. However, both the matched difference-in-differences analysis and the comparison between treatment and control communities at end-line suggested Watershared was not effective at reducing E. coli contamination at the landscape scale. Uptake of Watershared agreements was very low and the most important land from a water quality perspective (land around water intakes) was seldom enrolled. Discussion Although excluding cattle may have a positive local impact on water quality, higher uptake and better targeting would be required to achieve a significant impact on the quality of water consumed in the communities. Although RCTs potentially have an important role to play in building the evidence base for approaches such as PES, they are far from straightforward to implement. In this case, the randomised trial was not central to concluding that Watershared had not produced a landscape scale impact. We suggest that this RCT provides valuable lessons for future use of randomised experiments to evaluate landscape-scale environmental management interventions

Mechanisms and impacts of an incentive-based conservation program with evidence from a randomized control trial.

Conservation science needs more high-quality impact evaluations, especially ones that explore mechanisms of success or failure. Randomized control trials (RCTs) provide particularly robust evidence of the effectiveness of interventions (although they have been criticized as reductionist and unable to provide insights into mechanisms), but there have been few such experiments investigating conservation at the landscape scale. We explored the impact of Watershared, an incentive-based conservation program in the Bolivian Andes, with one of the few RCTs of landscape-scale conservation in existence. There is strong interest in such incentive-based conservation approaches as some argue they can avoid negative social impacts sometimes associated with protected areas. We focused on social and environmental outcomes based on responses from a household survey in 129 communities randomly allocated to control or treatment (conducted both at the baseline in 2010 and repeated in 2015-2016). We controlled for incomplete program uptake by combining standard RCT analysis with matching methods and investigated mechanisms by exploring intermediate and ultimate outcomes according to the underlying theory of change. Previous analyses, focused on single biophysical outcomes, showed that over its first 5 years Watershared did not slow deforestation or improve water quality at the landscape scale. We found that Watershared influenced some outcomes measured using the survey, but the effects were complex, and some were unexpected. We thus demonstrated how RCTs can provide insights into the pathways of impact, as well as whether an intervention has impact. This paper, one of the first registered reports in conservation science, demonstrates how preregistration can help make complex research designs more transparent, avoid cherry picking, and reduce publication bias

A global model of the response of tropical and sub-tropical forest biodiversity to anthropogenic pressures

Habitat loss and degradation, driven largely by agricultural expansion and intensification, present the greatest immediate threat to biodiversity. Tropical forests harbour among the highest levels of terrestrial species diversity and are likely to experience rapid land-use change in the coming decades. Synthetic analyses of observed responses of species are useful for quantifying how land use affects biodiversity and for predicting outcomes under land-use scenarios. Previous applications of this approach have typically focused on individual taxonomic groups, analysing the average response of the whole community to changes in land use. Here, we incorporate quantitative remotely sensed data about habitats in, to our knowledge, the first worldwide synthetic analysis of how individual species in four major taxonomic groups—invertebrates, ‘herptiles’ (reptiles and amphibians), mammals and birds—respond to multiple human pressures in tropical and sub-tropical forests. We show significant independent impacts of land use, human vegetation offtake, forest cover and human population density on both occurrence and abundance of species, highlighting the value of analysing multiple explanatory variables simultaneously. Responses differ among the four groups considered, and—within birds and mammals—between habitat specialists and habitat generalists and between narrow-ranged and wide-ranged species

The database of the PREDICTS (Projecting Responses of Ecological Diversity In Changing Terrestrial Systems) project

K

Can Payments for Ecosystem Services schemes reduce deforestation? A robust evaluation example from the Bolivian Andes.

There is growing interest in Payments for Ecosystem Services (PES) as a habitat conservation approach. Key questions remaining are 1) the extent to which conservation funded is additional (would not have occurred in absence of the scheme), and 2) the extent to which habitat degradation leaks away from enrolled land, and 3) whether improvements in outcomes of interest can be robustly attributed to the intervention. We explore these questions for Watershared, a widely implemented payment for watershed services scheme in Bolivia which incentivises upstream farmers to cease deforestation and exclude cattle from riparian forest to protect habitat and water quality. To robustly evaluate the effectiveness of the scheme at reducing deforestation, we made use of a Randomised Control Trial set-up, where 128 communities were divided randomly into treatment and control categories, and treatment community households were offered contracts on a voluntary basis. We evaluated additionality of enrolled land using 509 participating households’ responses to a survey on individual land parcels. Leakage was examined by evaluating deforestation at the community scale, thereby accounting for deforestation next to enrolled land. Intervention effect on deforestation was assessed using openly available Global Forest Change data to model the relationship between the PES scheme and control + treatment deforestation after implementation (2011-2016), accounting for competing predictors such as past deforestation (2000-2010) and geographical factors (e.g. slope, distance to road and river). We did not detect an effect of the scheme in an intention-to-treat analysis where control and treatment communities were included as a binary variable, which could plausibly be influenced by two factors. Firstly, only 13% of the area or 22% of contracts included in the survey were additional, showing a predominance of business-as-usual, and secondly, land enrolment across treatment communities was highly variable (0-80% of community area), limiting the use of binary analysis. Indeed, significant negative relationships between deforestation and % enrolled land were apparent both at the community (% deforestation) and the pixel (deforestation probability) scale of analysis (p=0.02, p<0.001), suggesting that the scheme reduces deforestation given adequate uptake. Importantly, this negative relationship also shows that leakage is not sufficient to offset conservation gains of the PES scheme. In a hypothetical scenario where the study communities all have 60% land enrolled, the gains in forest area would be ca 1000 ha compared with a 0% scenario, equating to >30% reduction in deforestation. We conclude that despite the small proportion of land enrolled being additional, measurable reductions in deforestation have occurred due to the PES scheme. Given the rapid spread in PES schemes but limited robust studies exploring their effectiveness, this research lends support to their wider use.peerReviewe

An Evaluation of the Impact of Payments for Ecosystem Services using a Randomized Control Trial

Payments for Ecosystem Services (PES) aim to incentivize land users to manage their land in ways which benefit society. However, as with many complex socio-ecological interventions, robust evaluation of PES is challenging and rare. Using a Randomized Control Trial (RCT) we evaluate whether a conservation program in the Bolivian Andes, which incentivizes landowners to avoid deforestation and exclude cattle from riparian forests, delivers improvements in microbial water quality (as measured by Escherichia coli contamination), and reduces deforestation. Baseline data was collected in 2010 in 129 communities which were then randomly allocated to a treatment or control group. The evaluation ran until 2016. Presence of cattle feces adversely affected microbial water quality, showing the effectiveness of excluding cattle, but the intervention itself did not have a demonstrable effect at the landscape scale. Using Global Forest Change data, we show that the intervention (especially at higher levels of uptake) does appear to reduce deforestation (once past deforestation and other geographical predictors are taken into account). Program effectiveness is fundamentally an empirical question so well-designed field studies are needed. We highlight some of the challenges encountered in using this pioneering RCT to robustly evaluate a large-scale conservation interventions but conclude that such randomized approaches have an important role to play in contributing to the evidence base available to decision-makers.peerReviewe