Rough Guide to Impact Evaluation of Environmental and Development Programs

This paper provides an overview of the what, how, and why of program evaluation, with particular emphasis on the role of control groups, pre-and post measurement, and covariate data. Resource and environmental economists in developing countries have little training in how to conduct programme or policy evaluations of proposed environment /sustainable development projects. The paper reviews detailed examples of four methods for evaluation: randomized experiments, natural experiments, matching methods, and panel-based “difference-in-difference” (DID) estimators with a description of the pros and cons of each method. It also provides a detailed case study from South Asia as an example

Benefit Transfer as Preference Calibration

This paper proposes and illustrates the use of a new approach to benefit transfer for the non-market valuation of environmental resources. It treats transfer as an identification problem that requires assessing whether available benefit estimates permit the parameters of a preference function to be identified. The transfer method proposed uses these identifying restrictions to calibrate preference parameters and bases the benefit estimates on that preference function. The approach is illustrated using travel cost, hedonic and contingent valuation estimates, as well as combinations of estimates. It has three potential advantages over conventional practice: (1) it allows multiple, potentially overlapping estimates of the benefits of an improvement in environmental quality to be combined consistently; (2) it assures the transferred estimates of the benefits attributed to a proposed change can never exceed income; and (3) it provides a set of additional "outputs" that offer plausibility checks of the benefit transfers.

Energy, gender and development: what are the linkages ? where is the evidence ?

This report reviews the literature on the links between energy access, welfare, and gender in order to provide evidence on where gender considerations in the energy sector matter and how they might be addressed. Prepared as a background document for the 2012 World Development Report on Gender Equality and Development, and part of the Social Development Department's ongoing work on gender and infrastructure, the report describes and evaluates the evidence on the links between gender and energy focusing on: increased access to woodfuel through planting of trees and forest management; improved cooking technologies; and access to electricity and motive energy. The report's main finding is that energy interventions can have significant gender benefits, which can be realized via careful design and targeting of interventions based on a context-specific understanding of energy scarcity and household decision-making, in particular how women's preferences, opportunity cost of time, and welfare are reflected in household energy decisions. The report focuses on the academic peer-reviewed literature and, although it applies fairly inclusive screening criteria when selecting the evidence to consider, finds that the evidence on many of the energy-gender linkages is often limited. There is thus a clear need for studies to evaluate interventions and identify key design elements for gender-sensitive project design.

Valuing water quality improvements in the United States using meta-analysis: Is the glass half-full or half-empty for national policy analysis?

The literature estimating the economic value for water quality changes has grown considerably over the last 30 years, resulting in an expanded pool of information potentially available to support national and regional policy analysis. Using 131 willingness to pay estimates from 18 studies that use a similar definition of water quality, we performed a meta-regression analysis and found mixed results. We find that WTP varies in systematic and expected ways with respect to factors such as the size of the water quality changes, average household income, and use/nonuse characteristics of respondents. As a whole, we conclude that our metaregression results provide a reasonable basis for estimating expected WTP values for defined changes in water quality. However, despite a large number of existing economic valuation studies, relatively few could be meaningfully combined through meta-analysis due to heterogeneity in the commodities being valued in the original studies. Based on these findings, we provide recommendations for future research, including suggestions regarding more standardized approaches for defining water quality and reporting information in valuation studies

Combining Revealed and Stated Preference Data to Estimate the Nonmarket Value of Ecological Services: An Assessment of the State of the Science

This paper reviews the marketing, transportation, and environmental economics literature on the joint estimation of revealed and stated preference data. The revealed preference and stated preference approaches are first described with a focus on the strengths and weaknesses of each. Recognizing these strengths and weaknesses, the potential gains from combining data are described. A classification system for combined data that emphasizes the type of data combination and the econometric models used is proposed. A methodological review of the literature is pursued based on this classification system. Examples from the environmental economics literature are highlighted. A discussion of the advantages and disadvantages of each type of jointly estimated model is then presented. Suggestions for future research, in particular opportunities for application of these models to environmental quality valuation, are presented.Nonmarket Valuation, Revealed Preference, Stated Preference

Preferences and the effectiveness of behavior-change interventions: Evidence from adoption of improved cookstoves in India

Preference heterogeneity can influence behavior in economically significant ways, thereby influencing the effectiveness of environmental policies or interventions. We test this hypothesis in the context of efficient cooking technology in India. We use stated preference methods to first characterize household tastes for various features of a more efficient cooking technology. We then relate these typically unobserved preferences to households' adoption decisions during an experiment that allowed them to choose between two alternatives with different features. Stated preferences help predict actual adoption: households initially classified as uninterested are less likely to purchase and use any new technology, while relative distaste for pollution is linked to selection of a cleaner technology. Because of this influence on adoption behaviors, preference heterogeneity has important implications for how environmental policies can impact various health and development outcomes

The use of willingness to pay experiments : estimating demand for piped water connections in Sri Lanka

The authors show how willingness to pay surveys can be used to gauge household demand for improved network water and sanitation services. They do this by presenting a case-study from Sri Lanka, where they surveyed approximately 1,800 households in 2003. Using multivariate regression, they show that a complex combination of factors drives demand for service improvements. While poverty and costs are found to be key determinants of demand, the authors also find that location, self-provision, and perceptions matter as well, and that subsets of these factors matter differently for subsamples of the population. To evaluate the policy implications of the demand analysis, they use the model to estimate uptake rates of improved service under various scenarios-demand in subgroups, the institutional decision to rely on private sector provision, and various financial incentives targeted to the poor. The simulations show that in this particular environment in Sri Lanka, demand for piped water services is low, and that it is unlikely that under the present circumstances the goal of nearly universal piped water coverage is going to be achieved. Policy instruments, such as subsidization of connection fees, could be used to increase demand for piped water, but it is unclear whether the benefits of the use of such policies would outweigh the costs.Town Water Supply and Sanitation,Environmental Economics&Policies,Water Use,Small Area Estimation Poverty Mapping,Urban Water Supply and Sanitation

Speaking from experience: preferences for cooking with biogas in rural India

Biogas has the potential to satisfy the clean energy needs of millions of households in under-served and energy-poor rural areas, while reducing both private and social costs linked to (i) fuels for household cooking, (ii) fertilizers, (iii) pressure on forests, and (iv) emissions (e.g., PM 2.5 and methane) that damage both household health and global climate. While the literature has focused on identifying these costs, less attention has been paid to household preferences for biogas systems — specifically what attributes are popular with which types of households. We conduct a discrete choice experiment with 503 households in rural Odisha, India, to better characterize preferences for different attributes (smoke reduction, fuel efficiency, and maintenance) and for different cooking technologies (biogas and an improved biomass cookstove). We find that on average households value smoke reduction and fuel efficiency. Willingness to pay (WTP) a premium for the improved biomass cookstove is low, while willingness to pay a premium for biogas is high. Nonetheless, WTP varies by the type of previous experience with biogas (e.g., good or bad experience) and with time and risk preferences of households. While risk-averse and impatient respondents have lower WTP for the improved cookstoves, previous experience with biogas attenuates this gap. These findings suggest that biogas uptake and diffusion could be improved by complementing existing subsidies with technology trials, good quality products, maintenance, and customer services to reduce uncertainty

Unpackaging demand for water service quality : evidence from conjoint surveys in Sri Lanka

In the early 2000s, the Government of Sri Lanka considered engaging private sector operators to manage water and sewerage services in two separate service areas: one in the town of Negombo (north of Colombo), and one stretching along the coastal strip (south from Colombo) from the towns of Kalutara to Galle. Since then, the government has abandoned the idea of setting up a public-private partnership in these two areas. This paper is part of a series of investigations to determine how these pilot private sector transactions (forming part of the overall water sector reform strategy) could be designed in such a manner that they would benefit the poor. The authors describe the results of a conjoint survey evaluating the factors that drive customer demand for alternative water supply and sanitation services in Sri Lanka. They show how conjoint surveys can be used to unpackage household demand for attributes of urban services and improve the design of infrastructure policies. They present conjoint surveys as a tool for field experiments and a source of valuable empirical data. In the study of three coastal towns in southwestern Sri Lanka the conjoint survey allows the authors to compare household preferences for four water supply attributes-price, quantity, safety, and reliability. They examine subpopulations of different income levels to determine if demand is heterogeneous. The case study suggests that households care about service quality (not just price). In general, the authors find that households have diverse preferences in terms of quantity, safety, and service options, but not with regard to hours of supply. In particular, they find that the poor have lower ability to trade off income for services, a finding that has significant equity implications in terms of allocating scarce public services and achieving universal water access.Town Water Supply and Sanitation,Water and Industry,Economic Theory&Research,Water Use,Water Supply and Sanitation Governance and Institutions