The politicisation of evaluation: constructing and contesting EU policy performance

Although systematic policy evaluation has been conducted for decades and has been growing strongly within the European Union (EU) institutions and in the member states, it remains largely underexplored in political science literatures. Extant work in political science and public policy typically focuses on elements such as agenda setting, policy shaping, decision making, or implementation rather than evaluation. Although individual pieces of research on evaluation in the EU have started to emerge, most often regarding policy “effectiveness” (one criterion among many in evaluation), a more structured approach is currently missing. This special issue aims to address this gap in political science by focusing on four key focal points: evaluation institutions (including rules and cultures), evaluation actors and interests (including competencies, power, roles and tasks), evaluation design (including research methods and theories, and their impact on policy design and legislation), and finally, evaluation purpose and use (including the relationships between discourse and scientific evidence, political attitudes and strategic use). The special issue considers how each of these elements contributes to an evolving governance system in the EU, where evaluation is playing an increasingly important role in decision making

Species identification by experts and non-experts: comparing images from field guides

Accurate species identification is fundamental when recording ecological data. However, the ability to correctly identify organisms visually is rarely questioned. We investigated how experts and non-experts compared in the identification of bumblebees, a group of insects of considerable conservation concern. Experts and non-experts were asked whether two concurrent bumblebee images depicted the same or two different species. Overall accuracy was below 60% and comparable for experts and non-experts. However, experts were more consistent in their answers when the same images were repeated, and more cautious in committing to a definitive answer. Our findings demonstrate the difficulty of correctly identifying bumblebees using images from field guides. Such error rates need to be accounted for when interpreting species data, whether or not they have been collected by experts. We suggest that investigation of how experts and non-experts make observations should be incorporated into study design, and could be used to improve training in species identification

Mission-oriented public policy and the new evaluation culture

In this chapter, our aim is to develop a framework to improve public policy-related evaluation practice for a more adaptive and anticipatory evaluation approach, better in tune with complex interactions and interdependencies that have emerged on our policy agenda today. One of the features of this space for interactions that is public policy is its mission orientation. Such an orientation is accompanied by the evolution of public policy instruments, which in turn necessitate new evaluation approaches. We are convinced that this requires developing a conceptual framework, which can be taken forward to test and further operationalise in situations where similar systemic transformations for policy development are elaborated upon. Based on our work on public-sector leadership, we are proposing a framework for evaluation in a more mission-driven and systems-based perspective. The framework seeks to take better into consideration the diversity of policy interventions at our disposal, ranging from traditional budgetary or legislative instruments to experimentation and piloting. Changes are identified in the very characteristics of the societal problems we are trying to solve, as well as in the nature of policy, both subsequently requiring a more multifaceted scope of evaluation, an emerging practice being towards a more mission-oriented one as well as a more nuanced approach depending on whether one is interested in the multi-organisational performance, policy service delivery or quality of outputs and impacts from policy initiatives and projects. The focus of evaluation in turn ranges from the accountability to evaluation criteria, timescale, motivation, as well as type of intervention used.fi=vertaisarvioitu|en=peerReviewed

Policy monitoring in the EU: The impact of institutions, implementation, and quality

Policy monitoring is often seen as a crucial ingredient of policy evaluation, but theoretically informed empirical analyses of real-world policy monitoring practices are still rare. This paper addresses this gap by focusing on climate policy monitoring in the European Union, which has a relatively stringent system of greenhouse gas monitoring but a much less demanding approach to monitoring policies. It explores how institutional settings, policy implementation, and the quality of information may impact the practices and politics of policy monitoring. Drawing on quantitative regression models and qualitative interviews, it demonstrates that policy monitoring has evolved over time and is itself subject to implementation pressures, but also exhibits learning effects that improve its quality. In further developing both everyday policy monitoring practices and academic understanding of them, there is a need to pay attention to their design—specifically, the impact of any overarching rules, the institutional support for implementation, and the criteria governing the quality of the information they deliver. In short, policy monitoring should be treated as a governance activity in its own right, raising many different design challenges

The Canadian Bandaging Trial: Evidence-informed leg ulcer care and the effectiveness of two compression technologies

Background: Objective: To determine the relative effectiveness of evidence-informed practice using two high compression systems: four-layer (4LB) and short-stretch bandaging (SSB) in community care of venous leg ulcers. Design and Setting: Pragmatic, multi-centre, parallel-group, open-label, randomized controlled trial conducted in 10 centres. Cognitively intact adults (≥18 years) referred for community care (home or clinic) with a venous ulceration measuring ≥0.7cm and present for ≥1 week, with an ankle brachial pressure index (ABPI) ≥0.8, without medication-controlled Diabetes Mellitus or a previous failure to improve with either system, were eligible to participate.Methods: Consenting individuals were randomly allocated (computer-generated blocked randomization schedule) to receive either 4LB or SSB following an evidence-informed protocol. Primary endpoint: time-to- healing of the reference ulcer. Secondary outcomes: recurrence rates, health-related quality of life (HRQL), pain, and expenditures.Results: 424 individuals were randomized (4LB n = 215; SSB n = 209) and followed until their reference ulcer was healed (or maximum 30 months). An intent-to-treat analysis was conducted on all participants. Median time to ulcer healing in the 4LB group was 62 days [95% confidence interval (CI) 51 to 73], compared with 77 days (95% CI 63 to 91) in the SSB group. The unadjusted Kaplan-Meier curves revealed the difference in the distribution of cumulative healing times was not significantly different between group (log rank χ2 = 0.001, P = 0.98) nor ulcers recurrence (4LB, 10.1%; SSB, 13.3%; p = 0.345). Multivariable Cox Proportional Hazard Modeling also showed no significant between-bandage differences in healing time after controlling for significant covariates (p = 0.77). At 3-months post-baseline there were no differences in pain (no pain: 4LB, 22.7%; SSB, 26.7%; p = 0.335), or HRQL (SF-12 Mental Component Score: 4LB, 55.1; SSB, 55.8; p = 0.615; SF-12 Physical Component Score: 4LB, 39.0; SSB, 39.6; p = 0.675). The most common adverse events experienced by both groups included infection, skin breakdown and ulcer deterioration.Conclusions: The Canadian Bandaging Trial revealed that in the practice context of trained RNs using an evidence-informed protocol, the choice of bandage system (4LB and SSB) does not materially affect healing times, recurrence rates, HRQL, or pain. From a community practice perspective, this is positive news for patient-centred care allowing individual/family and practitioner choice in selecting compression technologies based on circumstances and context.Trial registration: clinicaltrials.gov Identifier: NCT00202267

Surface Hardness Impairment of Quorum Sensing and Swarming for Pseudomonas aeruginosa

The importance of rhamnolipid to swarming of the bacterium Pseudomonas aeruginosa is well established. It is frequently, but not exclusively, observed that P. aeruginosa swarms in tendril patterns—formation of these tendrils requires rhamnolipid. We were interested to explain the impact of surface changes on P. aeruginosa swarm tendril development. Here we report that P. aeruginosa quorum sensing and rhamnolipid production is impaired when growing on harder semi-solid surfaces. P. aeruginosa wild-type swarms showed huge variation in tendril formation with small deviations to the “standard” swarm agar concentration of 0.5%. These macroscopic differences correlated with microscopic investigation of cells close to the advancing swarm edge using fluorescent gene reporters. Tendril swarms showed significant rhlA-gfp reporter expression right up to the advancing edge of swarming cells while swarms without tendrils (grown on harder agar) showed no rhlA-gfp reporter expression near the advancing edge. This difference in rhamnolipid gene expression can be explained by the necessity of quorum sensing for rhamnolipid production. We provide evidence that harder surfaces seem to limit induction of quorum sensing genes near the advancing swarm edge and these localized effects were sufficient to explain the lack of tendril formation on hard agar. We were unable to artificially stimulate rhamnolipid tendril formation with added acyl-homoserine lactone signals or increasing the carbon nutrients. This suggests that quorum sensing on surfaces is controlled in a manner that is not solely population dependent

Evaluation for what purpose? Findings from two stakeholder groups

A host of reasons exist for the pursuit of evidence in the public sector, including to support good governance and policy development. As the expectations for program evaluation from policymakers have evolved, so too has evaluation practice and a great deal of experimentalism has ensued. There is a risk that these developments and the complexity inherent in them, may lead to conflicting expectations about why program evaluation is done, or even a loss of purpose. This prompts the meso-level analysis of two types of stakeholders in a governance network, explored in this chapter. This chapter presents the findings of an ongoing study which explores the perceptions of program evaluators and policy implementers towards the purpose of evidence. The findings suggest that program evaluators and policy implementers have divergent expectations of why and how evaluation data might be used. The findings suggest that program evaluators aspire to support change and enhance the policy domains they serve, whereas policy implementers perceive program evaluation as serving a more governance-/management-orientated role. The chapter demonstrates the complexity of both program evaluation and policy and may have implications for the twin pillars of governance and responsibility at the heart of the book. If governance and responsibility are the twin pillars of sustainability then the complex networks of relationships, expectations, values, and outcomes may need to be considered. The findings also have implications for evaluation commissioners and practitioners, demonstrating the need for the purpose and expectations of program evaluation to be agreed early. The use of program evaluation as a symbolic, aesthetic or structural mechanism also emerges, prompting opportunity for further research, for instance, to explore legitimacy and program evaluation.N/

Cryptococcal meningitis: epidemiology, immunology, diagnosis and therapy.

HIV-associated cryptococcal meningitis is by far the most common cause of adult meningitis in many areas of the world that have high HIV seroprevalence. In most areas in Sub-Saharan Africa, the incidence of cryptococcal meningitis is not decreasing despite availability of antiretroviral therapy, because of issues of adherence and retention in HIV care. In addition, cryptococcal meningitis in HIV-seronegative individuals is a substantial problem: the risk of cryptococcal infection is increased in transplant recipients and other individuals with defects in cell-mediated immunity, and cryptococcosis is also reported in the apparently immunocompetent. Despite therapy, mortality rates in these groups are high. Over the past 5 years, advances have been made in rapid point-of-care diagnosis and early detection of cryptococcal antigen in the blood. These advances have enabled development of screening and pre-emptive treatment strategies aimed at preventing the development of clinical infection in patients with late-stage HIV infection. Progress in optimizing antifungal combinations has been aided by evaluation of the clearance rate of infection by using serial quantitative cultures of cerebrospinal fluid (CSF). Measurement and management of raised CSF pressure, a common complication, is a vital component of care. In addition, we now better understand protective immune responses in HIV-associated cases, immunogenetic predisposition to infection, and the role of immune-mediated pathology in patients with non-HIV associated infection and in the context of HIV-associated immune reconstitution reactions