Impact Evaluations and Development: Nonie Guidance on Impact Evaluation

In international development, impact evaluation is principally concerned with final results of interventions (programs, projects, policy measures, reforms) on the welfare of communities, households, and individuals, including taxpayers and voters. Impact evaluation is one tool within the larger toolkit of monitoring and evaluation (including broad program evaluations, process evaluations, ex ante studies, etc.).The Network of Networks for Impact Evaluation (NONIE) was established in 2006 to foster more and better impact evaluations by its membership -- the evaluation networks of bilateral and multilateral organizations focusing on development issues, as well as networks of developing country evaluators. NONIE's member networks conduct a broad set of evaluations, examining issues such as project and strategy performance, institutional development, and aid effectiveness. By sharing methodological approaches and promoting learning by doing on impact evaluations, NONIE aims to promote the use of this more specific approach by its members within their larger portfolio of evaluations. This document, by Frans Leeuw and Jos Vaessen, has been developed to support this focus.For development practitioners, impact evaluations play a keyrole in the drive for better evidence on results and development effectiveness. They are particularly well suited to answer important questions about whether development interventions do or do not work, whether they make a difference, and how cost-effective they are. Consequently, they can help ensure that scarce resources are allocated where they can have the most developmental impact

Evaluation Capture, Evaluator Resilience, and the Need for Competencies of Evaluators

Evaluation capture is the situation in which evaluations and evaluators are surrounded by protocols, guidelines, standards, norms, criteria, templates, oversight and reviews from national and supranational organizations, governments, NGO’s and agencies that ask for and finance monitoring and evaluation activities; these protocols and guidelines are sometimes labelled or part of ‘evaluation policies.’ The paper describes several developments in evaluation that contribute to this capture, referred to as mechanisms. The paper also addresses the question how to tackle this issue. The focus is on the role of the evaluator’s resilience, while several competences of evaluators to realize that are also addressed

The Performance Paradox in the Public Sector

Administrative reform has led to a strong increase in the use of performance assessment instruments in the public sector. However, this has also led to several unintended consequences, such as the performance paradox, tunnel vision and ‘analysis paralysis’. These unintended consequences can reduce the quality of the knowledge about actual levels of performance, or even negatively affect performance. Examples can be found in all policy sectors. We argue that certain characteristics of the public sector – such as ambiguous policy objectives, discretionary authority of street level bureaucrats, simultaneous production and consumption of services, and the disjunction of costs and revenues – increase the risk of a performance paradox, either unintentionally or deliberately. Performance assessment should therefore take the special characteristics of the public sector into account and develop systems that can handle contested and multiple performance indicators, striking a balance in the degree of ‘meas

First inventory of policy on counterterrorism: Germany, France, Italy, Spain, the United Kingdom and the United States - 'research in progress'

This report presents a first inventory of measures aimed at combating radicalisation, extremism and terrorism (referred to in this report as counterterrorism policy) in Germany, France, Italy, Spain, the United Kingdom and the United States. Commissioned by the NCTb, the WODC (the Research and Documentation Centre of the Dutch Ministry of Justice) carried out a study into counterterrorism measures in place in the countries investigated in early 2006. This study comprises the first international inventory of a number of policy fields under development, whereby the counterterrorism policies discussed here are characterised by extremely rapid development and innovation. This inventory can serve as point of departure for the registration of policy developments and to obtain an insight into their effectiveness. The continuous monitoring of the developments is therefore recommended. Researchers in Germany, France, Italy, Spain and the United States have drawn up working documents, each with an inventory of the policy in place in that particular country, taking their information from public sources, mostly government publications and Websites, as well as articles, reports and books. Data on the United Kingdom were collected by the WODC. The closing date for the data collection was 1 February 2006. Besides the inventory, the researchers were invited: - to present a brief history of the development of the counterterrorism policy in the country in question, focusing on experiences with past terrorism and how it was combated - to devote attention where possible to (policy) assumptions on which more recent measures are based - to present available evaluations. Due to the study’s character of providing an inventory, differences may occur between the countries as regards topic detailing. This is “work in progress” involving a policy field in progress. There is no systematic comparison between the countries. It is not possible, therefore, to draw conclusions based on this report for the introduction of certain measures in the Netherlands. However, following a summary of the key study results, seven strategic issues were identified as they emerged from the material

Proportions of B-cell subsets are altered in incomplete systemic lupus erythematosus and correlate with interferon score and IgG levels

OBJECTIVES: Incomplete SLE (iSLE) patients display symptoms typical for SLE but have insufficient criteria to fulfil the diagnosis. Biomarkers are needed to identify iSLE patients that will progress to SLE. IFN type I activation, B-cell-activating factor (BAFF) and B-cell subset distortions play an important role in the pathogenesis of SLE. The aim of this cross-sectional study was to investigate whether B-cell subsets are altered in iSLE patients, and whether these alterations correlate with IFN scores and BAFF levels. METHODS: iSLE patients (n = 34), SLE patients (n = 41) with quiescent disease (SLEDAI ≤4) and healthy controls (n = 22) were included. Proportions of B-cell subsets were measured with flow cytometry, IFN scores with RT-PCR and BAFF levels with ELISA. RESULTS: Proportions of age-associated B-cells were elevated in iSLE patients compared with healthy controls and correlated with IgG levels. In iSLE patients, IFN scores and BAFF levels were significantly increased compared with healthy controls. Also, IFN scores correlated with proportions of switched memory B-cells, plasma cells and IgG levels, and correlated negatively with complement levels in iSLE patients. CONCLUSION: In this cross-sectional study, distortions in B-cell subsets were observed in iSLE patients and were correlated with IFN scores and IgG levels. Since these factors play an important role in the pathogenesis of SLE, iSLE patients with these distortions, high IFN scores, and high levels of IgG and BAFF may be at risk for progression to SLE