Rough Guide to Impact Evaluation of Environmental and Development Programs

This paper provides an overview of the what, how, and why of program evaluation, with particular emphasis on the role of control groups, pre-and post measurement, and covariate data. Resource and environmental economists in developing countries have little training in how to conduct programme or policy evaluations of proposed environment /sustainable development projects. The paper reviews detailed examples of four methods for evaluation: randomized experiments, natural experiments, matching methods, and panel-based “difference-in-difference” (DID) estimators with a description of the pros and cons of each method. It also provides a detailed case study from South Asia as an example

Energy, gender and development: what are the linkages ? where is the evidence ?

This report reviews the literature on the links between energy access, welfare, and gender in order to provide evidence on where gender considerations in the energy sector matter and how they might be addressed. Prepared as a background document for the 2012 World Development Report on Gender Equality and Development, and part of the Social Development Department's ongoing work on gender and infrastructure, the report describes and evaluates the evidence on the links between gender and energy focusing on: increased access to woodfuel through planting of trees and forest management; improved cooking technologies; and access to electricity and motive energy. The report's main finding is that energy interventions can have significant gender benefits, which can be realized via careful design and targeting of interventions based on a context-specific understanding of energy scarcity and household decision-making, in particular how women's preferences, opportunity cost of time, and welfare are reflected in household energy decisions. The report focuses on the academic peer-reviewed literature and, although it applies fairly inclusive screening criteria when selecting the evidence to consider, finds that the evidence on many of the energy-gender linkages is often limited. There is thus a clear need for studies to evaluate interventions and identify key design elements for gender-sensitive project design.

Valuing water quality improvements in the United States using meta-analysis: Is the glass half-full or half-empty for national policy analysis?

The literature estimating the economic value for water quality changes has grown considerably over the last 30 years, resulting in an expanded pool of information potentially available to support national and regional policy analysis. Using 131 willingness to pay estimates from 18 studies that use a similar definition of water quality, we performed a meta-regression analysis and found mixed results. We find that WTP varies in systematic and expected ways with respect to factors such as the size of the water quality changes, average household income, and use/nonuse characteristics of respondents. As a whole, we conclude that our metaregression results provide a reasonable basis for estimating expected WTP values for defined changes in water quality. However, despite a large number of existing economic valuation studies, relatively few could be meaningfully combined through meta-analysis due to heterogeneity in the commodities being valued in the original studies. Based on these findings, we provide recommendations for future research, including suggestions regarding more standardized approaches for defining water quality and reporting information in valuation studies

Combining Revealed and Stated Preference Data to Estimate the Nonmarket Value of Ecological Services: An Assessment of the State of the Science

This paper reviews the marketing, transportation, and environmental economics literature on the joint estimation of revealed and stated preference data. The revealed preference and stated preference approaches are first described with a focus on the strengths and weaknesses of each. Recognizing these strengths and weaknesses, the potential gains from combining data are described. A classification system for combined data that emphasizes the type of data combination and the econometric models used is proposed. A methodological review of the literature is pursued based on this classification system. Examples from the environmental economics literature are highlighted. A discussion of the advantages and disadvantages of each type of jointly estimated model is then presented. Suggestions for future research, in particular opportunities for application of these models to environmental quality valuation, are presented.Nonmarket Valuation, Revealed Preference, Stated Preference

The use of willingness to pay experiments : estimating demand for piped water connections in Sri Lanka

The authors show how willingness to pay surveys can be used to gauge household demand for improved network water and sanitation services. They do this by presenting a case-study from Sri Lanka, where they surveyed approximately 1,800 households in 2003. Using multivariate regression, they show that a complex combination of factors drives demand for service improvements. While poverty and costs are found to be key determinants of demand, the authors also find that location, self-provision, and perceptions matter as well, and that subsets of these factors matter differently for subsamples of the population. To evaluate the policy implications of the demand analysis, they use the model to estimate uptake rates of improved service under various scenarios-demand in subgroups, the institutional decision to rely on private sector provision, and various financial incentives targeted to the poor. The simulations show that in this particular environment in Sri Lanka, demand for piped water services is low, and that it is unlikely that under the present circumstances the goal of nearly universal piped water coverage is going to be achieved. Policy instruments, such as subsidization of connection fees, could be used to increase demand for piped water, but it is unclear whether the benefits of the use of such policies would outweigh the costs.Town Water Supply and Sanitation,Environmental Economics&Policies,Water Use,Small Area Estimation Poverty Mapping,Urban Water Supply and Sanitation

Speaking from experience: preferences for cooking with biogas in rural India

Biogas has the potential to satisfy the clean energy needs of millions of households in under-served and energy-poor rural areas, while reducing both private and social costs linked to (i) fuels for household cooking, (ii) fertilizers, (iii) pressure on forests, and (iv) emissions (e.g., PM 2.5 and methane) that damage both household health and global climate. While the literature has focused on identifying these costs, less attention has been paid to household preferences for biogas systems — specifically what attributes are popular with which types of households. We conduct a discrete choice experiment with 503 households in rural Odisha, India, to better characterize preferences for different attributes (smoke reduction, fuel efficiency, and maintenance) and for different cooking technologies (biogas and an improved biomass cookstove). We find that on average households value smoke reduction and fuel efficiency. Willingness to pay (WTP) a premium for the improved biomass cookstove is low, while willingness to pay a premium for biogas is high. Nonetheless, WTP varies by the type of previous experience with biogas (e.g., good or bad experience) and with time and risk preferences of households. While risk-averse and impatient respondents have lower WTP for the improved cookstoves, previous experience with biogas attenuates this gap. These findings suggest that biogas uptake and diffusion could be improved by complementing existing subsidies with technology trials, good quality products, maintenance, and customer services to reduce uncertainty

Unpackaging demand for water service quality : evidence from conjoint surveys in Sri Lanka

In the early 2000s, the Government of Sri Lanka considered engaging private sector operators to manage water and sewerage services in two separate service areas: one in the town of Negombo (north of Colombo), and one stretching along the coastal strip (south from Colombo) from the towns of Kalutara to Galle. Since then, the government has abandoned the idea of setting up a public-private partnership in these two areas. This paper is part of a series of investigations to determine how these pilot private sector transactions (forming part of the overall water sector reform strategy) could be designed in such a manner that they would benefit the poor. The authors describe the results of a conjoint survey evaluating the factors that drive customer demand for alternative water supply and sanitation services in Sri Lanka. They show how conjoint surveys can be used to unpackage household demand for attributes of urban services and improve the design of infrastructure policies. They present conjoint surveys as a tool for field experiments and a source of valuable empirical data. In the study of three coastal towns in southwestern Sri Lanka the conjoint survey allows the authors to compare household preferences for four water supply attributes-price, quantity, safety, and reliability. They examine subpopulations of different income levels to determine if demand is heterogeneous. The case study suggests that households care about service quality (not just price). In general, the authors find that households have diverse preferences in terms of quantity, safety, and service options, but not with regard to hours of supply. In particular, they find that the poor have lower ability to trade off income for services, a finding that has significant equity implications in terms of allocating scarce public services and achieving universal water access.Town Water Supply and Sanitation,Water and Industry,Economic Theory&Research,Water Use,Water Supply and Sanitation Governance and Institutions

Money for Nothing? A Call for Empirical Evaluation of Biodiversity Conservation Investments

The field of conservation policy must adopt state-of-the-art program evaluation methods to determine what works, and when, if we are to stem the global decline of biodiversity and improve the effectiveness of conservation investments

Implementation Science to Accelerate Clean Cooking for Public Health

Clean cooking has emerged as a major concern for global health and development because of the enormous burden of disease caused by traditional cookstoves and fires. The World Health Organization has developed new indoor air quality guidelines that few homes will be able to achieve without replacing traditional methods with modern clean cooking technologies, including fuels and stoves. However, decades of experience with improved stove programs indicate that the challenge of modernizing cooking in impoverished communities includes a complex, multi-sectoral set of problems that require implementation research. The National Institutes of Health, in partnership with several government agencies and the Global Alliance for Clean Cookstoves, has launched the Clean Cooking Implementation Science Network that aims to address this issue. In this article, our focus is on building a knowledge base to accelerate scale-up and sustained use of the cleanest technologies in low- and middle-income countries. Implementation science provides a variety of analytical and planning tools to enhance effectiveness of clinical and public health interventions. These tools are being integrated with a growing body of knowledge and new research projects to yield new methods, consensus tools, and an evidence base to accelerate improvements in health promised by the renewed agenda of clean cooking.Fil: Rosenthal, Joshua. National Institutes Of Health. Fogarty International Center; Estados UnidosFil: Balakrishnan, Kalpana. Sri Ramachandra University; IndiaFil: Bruce, Nigel. University of Liverpool; Reino UnidoFil: Chambers, David. National Institutes of Health. National Cancer Institute; Estados UnidosFil: Graham, Jay. The George Washington University; Estados UnidosFil: Jack, Darby. Columbia University; Estados UnidosFil: Kline, Lydia. National Institutes Of Health. Fogarty International Center; Estados UnidosFil: Masera, Omar Raul. Universidad Nacional Autónoma de México; MéxicoFil: Mehta, Sumi. Global Alliance for Clean Cookstoves; Estados UnidosFil: Mercado, Ilse Ruiz. Universidad Nacional Autónoma de México; MéxicoFil: Neta, Gila. National Institutes of Health. National Cancer Institute; Estados UnidosFil: Pattanayak, Subhrendu. University of Duke; Estados UnidosFil: Puzzolo, Elisa. Global LPG Partnership; Estados UnidosFil: Petach, Helen. U.S. Agency for International Development; Estados UnidosFil: Punturieri, Antonello. National Heart, Lung, and Blood Institute; Estados UnidosFil: Rubinstein, Adolfo Luis. Instituto de Efectividad Clínica y Sanitaria; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Sage, Michael. Centers for Disease Control and Prevention; Estados UnidosFil: Sturke, Rachel. National Institutes Of Health. Fogarty International Center; Estados UnidosFil: Shankar, Anita. University Johns Hopkins; Estados UnidosFil: Sherr, Kenny. University of Washington; Estados UnidosFil: Smith, Kirk. University of California at Berkeley; Estados UnidosFil: Yadama, Gautam. Washington University in St. Louis; Estados Unido