Economic impacts of conditional cash transfer programmes: a systematic review and meta-analysis

The results of a systematic review of evidence on the effects of conditional cash transfer (CCT) programmes on household economic outcomes are presented. Out of 1076 original articles found through electronic and handsearches, 46 randomised and quasi-experimental impact evaluations were eligible for the review. The authors used statistical meta-analysis and analysis of programme mechanisms to explore heterogeneity in impacts between and within programmes. They conclude that, for households which benefited from those CCT programmes which have been rigorously evaluated, child labour decreased, particularly for boys, household consumption and investment increased and consumption smoothing improved. In addition, there were limited effects on girls’ labour and mixed effects on adult labour supply in beneficiary households. Limited evidence has been collected on locality-wide impacts in beneficiary communities

Maintaining Momentum to 2015? An impact evaluation of interventions to improve maternal and child health and nutrition in Bangladesh

Bangladesh has experienced rapid fertility decline and reductions in under-five mortality over the last three decades. This impact study unravels the various factors behind these changes. Economic growth has been important, but so have major public sector interventions, notably reproductive health and immunization, supported by external assistance from the World Bank and other agencies. By contrast, nutrition began to improve only in the 1990s and remains high. The Bangladesh Integrated Nutrition Program (BINP) has played a small role, if any, in this progress, which is mainly attributable to higher agricultural productivity.Bangladesh, mortality, fertility, nutrition, health, population

Quasi-experimental study designs series –Paper 9: Collecting Data from Quasi-Experimental Studies

Objective: To identify variables that must be coded when synthesizing primary studies that use quasi-experimental designs.  Study Design and Setting: All quasi-experimental (QE) designs.  Results: When designing a systematic review of QE studies potential sources of heterogeneity – both theory-based and methodological – must be identified. We outline key components of inclusion criteria for syntheses of quasi-experimental studies. We provide recommendations for coding content-relevant and methodological variables, and outlined the distinction between bivariate effect sizes and partial (i.e., adjusted) effect sizes. Designs used and controls employed are viewed as of greatest importance. Potential sources of bias and confounding are also addressed.  Conclusion: Careful consideration must be given to inclusion criteria and the coding of theoretical and methodological variables during the design phase of a synthesis of quasi-experimental studies. The success of the meta-regression analysis relies on the data available to the meta-analyst. Omission of critical moderator variables (i.e., effect modifiers) will undermine the conclusions of a meta-analysis

Quasi-experimental study designs series-paper 5: a checklist for classifying studies evaluating the effects on health interventions-a taxonomy without labels.

OBJECTIVES: The aim of the study was to extend a previously published checklist of study design features to include study designs often used by health systems researchers and economists. Our intention is to help review authors in any field to set eligibility criteria for studies to include in a systematic review that relate directly to the intrinsic strength of the studies in inferring causality. We also seek to clarify key equivalences and differences in terminology used by different research communities. STUDY DESIGN AND SETTING: Expert consensus meeting. RESULTS: The checklist comprises seven questions, each with a list of response items, addressing: clustering of an intervention as an aspect of allocation or due to the intrinsic nature of the delivery of the intervention; for whom, and when, outcome data are available; how the intervention effect was estimated; the principle underlying control for confounding; how groups were formed; the features of a study carried out after it was designed; and the variables measured before intervention. CONCLUSION: The checklist clarifies the basis of credible quasi-experimental studies, reconciling different terminology used in different fields of investigation and facilitating communications across research communities. By applying the checklist, review authors' attention is also directed to the assumptions underpinning the methods for inferring causality

The effectiveness of school-based decision making in improving educational outcomes: a systematic review

The rhetoric around decentralisation suggests school-based management improves education outcomes. Existing reviews on school-based decision-making have tended to focus on proximal outcomes and offer very little information about why school-based decision-making has positive or negative effects in different circumstances. The authors systematically searched for and synthesised evidence from 35 quantitative and qualitative studies evaluating 17 individual interventions on the effectiveness of school-based decision-making on educational outcomes. Devolving decision-making to the level of the school appears to have a somewhat beneficial effect on dropout, repetition and teacher attendance. Effects on test-scores are more robust, being positive in aggregate and for middle-income countries specifically. On the other hand, school-based decision-making reforms appear to be less effective in communities with generally low levels of education, where parents have low status relative to school personnel. The authors conclude that school-based decision-making reforms are less likely to be successful in highly disadvantaged communities

Can Non-Randomised Studies of Interventions Provide Unbiased Effect Estimates? A Systematic Review of Internal Replication Studies.

Non-randomized studies of intervention effects (NRS), also called quasi-experiments, provide useful decision support about development impacts. However, the assumptions underpinning them are usually untestable, their verification resting on empirical replication. The internal replication study aims to do this by comparing results from a causal benchmark study, usually a randomized controlled trial (RCT), with those from an NRS conducted at the same time in the sampled population. We aimed to determine the credibility and generalizability of findings in internal replication studies in development economics, through a systematic review and meta-analysis. We systematically searched for internal replication studies of RCTs conducted on socioeconomic interventions in low- and middle-income countries. We critically appraised the benchmark randomized studies, using an adapted tool. We extracted and statistically synthesized empirical measures of bias. We included 600 estimates of correspondence between NRS and benchmark RCTs. All internal replication studies were found to have at least "some concerns" about bias and some had high risk of bias. We found that study designs with selection on unobservables, in particular regression discontinuity, on average produced absolute standardized bias estimates that were approximately zero, that is, equivalent to the estimates produced by RCTs. But study conduct also mattered. For example, matching using pre-tests and nearest neighbor algorithms corresponded more closely to the benchmarks. The findings from this systematic review confirm that NRS can produce unbiased estimates. Authors of internal replication studies should publish pre-analysis protocols to enhance their credibility

Quasi-experimental study designs series-paper 6: risk of bias assessment.

OBJECTIVES: Rigorous and transparent bias assessment is a core component of high-quality systematic reviews. We assess modifications to existing risk of bias approaches to incorporate rigorous quasi-experimental approaches with selection on unobservables. These are nonrandomized studies using design-based approaches to control for unobservable sources of confounding such as difference studies, instrumental variables, interrupted time series, natural experiments, and regression-discontinuity designs. STUDY DESIGN AND SETTING: We review existing risk of bias tools. Drawing on these tools, we present domains of bias and suggest directions for evaluation questions. RESULTS: The review suggests that existing risk of bias tools provide, to different degrees, incomplete transparent criteria to assess the validity of these designs. The paper then presents an approach to evaluating the internal validity of quasi-experiments with selection on unobservables. CONCLUSION: We conclude that tools for nonrandomized studies of interventions need to be further developed to incorporate evaluation questions for quasi-experiments with selection on unobservables

The effect of microcredit on women's control over household spending in developing countries: a systematic review

Background: Over the past three decades, microfinance activities have spread across the globe, reaching tens of millions of poor households with tailored financial services. Microfinance can best be described as a field of intervention rather than a particular instrument. Initially, microfinance usually meant microcredit for working capital and very small investments, but increasingly it has been broadened to include savings/deposits, a limited range of micro-insurance and payment services (including micro-leasing) as well as a somewhat broader range of credit products for more substantial investments. In this study we focused on microcredit activities, constituting the bulk of microfinance activities across the globe. Microcredit activities have affected the lives of clients and others in multiple ways. The most frequently reported types of effects of credit at individual, enterprise and household level are the following: income, expenditure smoothing, and poverty alleviation effects; business growth and employment effects; schooling effects; and effects in terms of women's empowerment. Despite the diversity in microcredit schemes, many share two characteristics: they target poor women and often rely on some type of group-based lending. Women's empowerment in relation to microcredit has been studied extensively within the context of this type of microcredit scheme. Most of these studies have been carried out in the context of microcredit group schemes in South Asia. It has been argued that access to microcredit can foster changes in individual attitudes of women (e.g. increased self-reliance), power relations within the household (e.g. control over resources) and social status. An important dimension of empowerment concerns women's control over household spending. The main assumption is that by providing credit to poor women, their direct control over expenditures within the household increases, with subsequent implications for the status of women and the well-being of women and other household members. Women's control over household spending is a frequently recurring aspect analyzed within the context of microcredit interventions, which allows us to study whether microcredit targeted at women affects women's control over household spending decisions and the circumstances in which this occurs. Despite the central and recurrent role across studies of this aspect of women's empowerment in relation to microcredit activities, there has been no previous review on this topic. The growing importance of microcredit has resulted in a vast number of research and evaluation studies, including impact studies. Consequently, the microfinance literature harbors a substantial number of synthesis studies which discuss a set of microcredit interventions and aim to generate overall conclusions on their effects. However, most of these studies face limitations in terms of depth of empirical assessment and the extent to which the identified effects can be attributed to microcredit. Moreover, methodological principles regarding comprehensive searches and principles of selection, coding, extraction and aggregation are often lacking in review studies. Partial exceptions are three recent systematic reviews which all differ in scope from the present one (Stewart et al., 2010; Duvendack et al. 2011; Stewart et al., 2012). The reviews respectively focus on microfinance (credit and savings) in Sub-Sahara Africa, microcredit worldwide, and microfinance worldwide (credit, saving and leasing). Overall, these reviews suggest that the effects of microcredit on women's empowerment are at best mixed. In part this can be explained by the heterogeneity in microcredit interventions, contexts and target groups. However, the existing reviews did not use statistical meta-analysis to synthesise evidence of effects, nor context-mechanism-outcome synthesis to understand the variation in effects. Objectives: The main objective of this study was to provide a systematic review of the evidence on the effects of microcredit on women's control over household spending in developing countries. More specifically, we aimed to answer two related research questions: 1) what does the impact evaluative evidence say about the causal relationship between microcredit and specific dimensions of women's empowerment (women'ss control over household spending); and 2) what are the mechanisms which mediate this relationship. We prioritise depth of analysis over breadth, thus the scope of this review is narrower than previous systematic reviews on microfinance (Stewart et al., 2010; Duvendack et al. 2011; Stewart et al., 2012). We focused on specific aspects of women's empowerment which allowed us to combine statistical meta-analysis and realist (context-mechanism-outcome) synthesis. Criteria for considering studies for this review: We included studies that analyzed the effects of microcredit schemes targeting poor women in low and middle income countries, as defined by the World Bank. Studies that did not include analysis on microcredit and the effect on one or more dimensions (specified in main body of the report) of women's control over household expenditures were excluded. Studies which gave evidence of addressing the attribution problem either through randomised design, quasi-experimental matching, or regression analysis, were included. In practice, women's control over household spending (as a key dimension of empowerment) is influenced by many different factors. By focusing on those studies which explicitly addressed the challenge of separating the effect of microcredit from other influencing factors, we developed what we consider to be the most credible evidence base for drawing conclusions about the effects of microcredit on women's control over household expenditures in different contexts. SEARCH STRATEGY We conducted a comprehensive search covering all relevant academic databases, internet search engines and web sites with published and unpublished research, and also carried out extensive manual searches of books and additional journals not included in electronic data bases (searches were concluded on December 31, 2011). We used back-referencing from recent studies as well as citation-tracking to identify additional relevant studies. Finally, authors of studies which we were unable to retrieve were contacted. In addition, we contacted experts on microcredit and women's empowerment for additional references which we might have missed. Search strategies in databases and journals were adapted for each source. Where possible we used the existing keyword indices of particular databases. In addition, we applied our own list of combinations of keywords covering all relevant terms relating to the independent variable (i.e. credit and its variations) and the dependent variable (i.e. dimensions of women's control over household spending, empowerment). Data collection and analysis: From the different searches we identified an initial number of 310 papers that were selected for full text examination. Eventually, 29 papers were retained for further analysis, corresponding to 25 unique studies. These 25 independent findings were included in the synthesis. However, based on a systematic risk of bias assessment we found that more than half of the included studies had high threats to internal validity. Moreover, only about half of the studies show a clear and coherent link between a theoretical framework on microcredit and women's control over household spending and empirical data analysis. It should be noted that reviewing and synthesizing quantitative results from studies is only one side of the coin. The other side is to understand what makes them work, or what prevents them from working. Consequently, we conducted a qualitative synthesis of the included studies, which focused on identifying the mechanisms which underlie the causal relationship between microcredit and women's control over household spending. RESULTS The results of the meta-analysis indicated that the effect sizes from experimental studies examining effects of microcredit on women's control over household spending are not statistically significantly different from zero. The effects from quasi-experimental studies are statistically insignificant overall, and at best of small magnitude for those studies assessed of being of high risk of bias. We conclude that there is no consistent evidence for an effect of microcredit on women's control over household spending. In the qualitative analysis, using Coleman's (1986, 1990) typology of mechanisms, we identified five different situational mechanisms and eight different action-formation mechanisms. Due to the combination of substantial heterogeneity in contexts (e.g. existing gender relations) and interventions (e.g. microcredit versus microcredit and additional services), and the lack of information in the studies on this heterogeneity, it was not possible to go beyond the identification of mechanisms, in terms of generating empirically tested articulated theories of change which are representative beyond a specific study context. Authors' conclusions: In line with three recent other reviews on microfinance (Stewart et al., 2010; Duvendack et al., 2011; Stewart et al. 2012) we found that the microcredit evidence base is extensive, yet most studies are weak methodologically. From those studies deemed comparable and of minimum acceptable quality, we concluded that overall there is no evidence for an effect of microcredit on women's control over household spending. Women's control over household resources constitutes an important intermediary dimension in processes of women's empowerment. Given the overall lack of evidence for an effect of microcredit on women's control over household resources it is therefore very unlikely that, overall, microcredit has a meaningful and substantial impact on empowerment processes in a broader sense. While impacts on empowerment may appear to have occurred in particular studies, the high risk of bias of studies providing positive assessments suggests that such findings are of limited validity. Our conclusions on the effects of microcredit on empowerment are also in line with previous systematic reviews by Duvendack et al. (2011) and Stewart (et al. 2010) who report to a limited extent on empowerment effects. Consequently, there appears to be a gap between the often optimistic societal belief in the capacity of microcredit to ameliorate the position of women in decision-making processes within the household on the one hand, and the empirical evidence base on the other hand. However, our review markedly differs from previous reviews in two regards. First, we specifically focused on microcredit and women's empowerment captured through women's control over household expenditures. Second, as a result of this narrower focus, we were able to conduct statistical meta-analysis and extract behavioral mechanisms which can help to explain why and how microcredit can make a difference. The advantage of our approach was that the identified mechanisms all stem from studies which show evidence of addressing the attribution problem. Consequently, we can be quite confident of the insights that they provided on the effects of microcredit on women's control over household spending for particular populations of microcredit female clients and their families. Those studies that showed evidence of addressing the attribution problem were relatively weak on underlying theory. Moreover, they often lacked essential information such as the nature of the intervention and how it related to empowerment (e.g. how solidarity groups affect empowerment processes) or the slowly evolving gender relations in different contexts (e.g. the evolution of societal norms and the relationship with power relations in the household). A next logical step would be to undertake a systematic review of qualitative studies which often provide rich and context-specific information on microcredit and women's decision-making power in the household. Such a review should ideally build on the mechanisms identified in the present review and would bring us closer to uncovering credible theories of microcredit and the circumstances in which it may change women's decision-making power

Quantifying the effectiveness and health co-benefits of climate change mitigation actions across sectors: a protocol for an umbrella review

Background: Effective and rapid actions are required to achieve global goals for climate change mitigation, and there is an opportunity to ensure that the actions taken are also positive for human health. However, little is known about the relative magnitude of the health co-benefits that can be achieved from mitigation actions, so robust and comprehensive syntheses of the evidence on the nature and effects of relevant actions are required. This paper presents a protocol for an interdisciplinary and cross-sectoral umbrella review of systematic reviews, synthesising modelled and empirical evidence on such actions. Methods: Nine bibliographic databases will be searched, capturing literature across a wide range of disciplines and sectors. Unique records retrieved by the searches will be screened by two independent reviewers. The quality of all the included systematic reviews will be assessed using A MeaSurement Tool to Assess Systematic Reviews (AMSTAR) 2 critical appraisal tool. Data will be extracted on methodological and thematic characteristics of the reviews, nature of the actions, and their effects on greenhouse gas emission reduction, health, and its determinants, as well as any other reported effects and interactions across different actions. Results: Narrative and quantitative synthesis methods will be used to create a typology of relevant actions, map pathways to their impacts on health, compare the magnitude of health and greenhouse gas (GHG) emission reduction impacts by selected characteristics of the actions and the nature of the evidence, as well as to identify gaps in evidence syntheses. Conclusion: This review will identify the most effective actions for global climate change mitigation and health based on the best available scientific evidence.   This protocol has been registered in PROSPERO, Reg No.: CRD42021239292.</ns4:p