Beleidsevaluaties tussen methode en praktijk: Naar een meer realistische evaluatiebenadering

__Abstract__ Het idee dat evaluatie kan bijdragen aan het verbeteren van beleid is niets nieuws onder de zon. Al in de jaren ’80 van de vorige eeuw bloeide evaluatie op met de bedoeling beleidsformulering en beleidsuitvoering te informeren, en in het hedendaagse ‘evidence-based’ tijdperk is evaluatie meer dan ooit een bloeiende praktijk. 1 Nederland volgde historisch weliswaar een ander en sneller traject dan Belgie¨, waar evaluatie slechts na de millenniumwisseling algemeen ingang vond.2 De potentie van evaluatie is tegenwoordig niettemin grotendeels (h)erkend en wordt gevat in adagia als ‘leren van evalueren’. Er worden met regelmaat evaluatiestudies uitgevoerd, maar opvallend is dat deze in toenemende mate ook het gebruik van evaluatieonderzoek zelf betreffen

Validating Methods for Comparing Public Policy: Perspectives from Academics and "Pracademics". Introduction to the Special Issue Introduction

The politics and administration of institutional chang

Outsource versus in-house. An identification of the organizational conditions influencing the choice for internal or external evaluators

An evaluation can be conducted in-house or can be outsourcedto an external party. Yet organizations do not always have fulldiscretion to decide on the locus for evaluation implementation.Certain attributes often push the organization in one directionor another. Via a systematic pairwise comparison of attributes of18 organizations in the Flemish (Belgian) public sector, we wereable to indicate the conditions that matter most in determiningthe locus of policy evaluation implementation. Our findings canthus enrich existing guidelines on the advantages and disadvantagesof internal and external evaluations.The politics and administration of institutional chang

The knowledge behind Brexit. A bibliographic analysis of ex-ante policy appraisals on Brexit in the United Kingdom and the European Union

This is the final version. Available on open access from Taylor & Francis via the DOI in this recordIn this article we map and explain the sources of knowledge cited on 85 Brexit impact appraisals, 46 of which were formal impact assessments ordered and published by the European Parliament and 39 ‘sectoral reports’ ordered by the UK Government and released by the House of Commons Exiting the EU Committee. All reports were published between the day after the UK referendum and the year after the start of the UK-EU negotiations. We conducted a citation analysis of 3537 references and tested author push and policy sector pull hypotheses with non-parametric tests. Our findings highlight the epistemic function of the professional referent groups to which authors belong. Authors tend to generate information and cite sources that are congruent with their ‘home group’ in the departmental unit where they work, or their larger professional group, even in urgent high-salient risk situations like Brexit. Differences between policy sectors do not strongly matter

A capacity assessment framework for the fit-for-purpose land administration systems: the use of unmanned aerial vehicle (UAV) in Rwanda and Kenya

This article presents a novel capacity assessment framework, coined as Fit-For-Purpose capacity assessment framework (FCAF), to measure the capacity of the land administration system compliant with the Fit-For-Purpose approach. The framework incorporates legal, political, operational, social, technical, and technological capacity conditions and provides a holistic view of the capacity development pathways. The FCAF is designed by merging six capacity dimensions, namely regulations, political system, operational unit, social norms, land recording techniques, and software. FCAF systematically identifies context-specific, enabling and impeding capacity components and thus provides a basis to develop the necessary capacity development strategies and interventions. Specifically, FCAF can serve as a useful heuristic for the development of the capacity development strategies for the adaptation and sustainability of the geospatial technologies in land administration systems. In the article, by assessing the capacity needs for the adaptation of unmanned aerial vehicle (UAV) technology in Rwandese and Kenyan land administration systems, the efficacy of the FCAF is tested. The findings suggest that in Rwanda, capacity conditions are more supportive of an easier uptake of UAV. Nonetheless, weak market conditions and strict regulations concerning UAV call for attention. In Kenya, existing institutional and political challenges in the land administration system raise concerns about the reliability and attainability of UAV under the current framework conditions. Despite that, there are more supportive market conditions in Kenya in comparison to Rwanda and multiple non-governmental and private actors that can bolster the adaptation process into a more sustainable and scalable land administration system. The politics and administration of institutional chang

Impact of performance audit on the Administration: A Belgian study (2005-2010)

Purpose This study of the impact of Belgian Court of Audit on the federal Administration for the 2005 to 2010 period aims to highlight the auditors’ influence on the management of governmental organizations through the performance audits they have been conducting since 1998. A set of ten variables allows us to measure the three types of uses of performance auditors’ work by auditees: instrumental, conceptual and strategic uses. Design/methodology/approach A survey was sent out to a total of 148 respondents identified by the authorities of the targeted organizations. 47 usable questionnaires were completed (32% response rate). Findings The Court of Audit’s impact on the audited entities did not provoke radical changes in the auditees’ organizational life but the intervention of the auditors was nevertheless noticeable. The nature of the impact was rather conceptual than strategic or instrumental. And the negative consequences on auditees anticipated in the literature were not observed. Research limitations/implications Given the five-year period covered by the study which was made in 2014 (four years after 2010), it had to deal with the mortality of respondents and the loss of organizational memory. Practical implications The study gives more accurate insights about the influence that Supreme Audit Institutions actually exert on audited Administrations through their performance audits. Originality/value Since Supreme Audit Institutions have been mandated to evaluate government’s economy, efficiency and effectiveness for almost 40 years in the western democracies, it is mandatory that their actual ability to influence Administrations be documented more abundantly and independently by academic researchers.  The politics and administration of institutional chang

Academic policy advice in consensus-seeking countries: the cases of Belgium and Germany

Research on policy-advisory systems worldwide has shown that historically dominant sources of advice traditionally located in-house to the government have been increasingly supplemented by other actors and outside knowledge. However, the vast majority of research has concentrated on the anglophone context. Yet, countries with a consensus-seeking, neo-corporatist tradition provide a special case in terms of policy advice and merit more scholarly attention. What counts as evidence in these countries is the expert rationality of institutional representatives. The position and role of academic research in consensus-based systems is unclear, and is the focus of this article. Can we observe commonalities across consensus-style countries, or do differences prevail? We investigate two typical consensus-seeking countries: Belgium and Germany. To examine the supply side of policy advice, the article reviews current evidence regarding their policy-advisory systems. For the demand side, we present insights from a survey among federal ministerial officials. We find common trends between the two cases but their nature and extent are idiosyncratic. In Belgium, the supply of and demand for academic policy advice is comparatively lower, while the German case exhibits more change in the advisory landscape and institutionalisation of the supply of and demand for academic research. The politics and administration of institutional chang

Conservative solutions for progress: on solution types when combining QCA with in-depth Process-Tracing

What is the most appropriate QCA solution type when engaging in a multimethod design that includes QCA and in-depth process-tracing (PT)? While either the intermediate or the parsimonious solution are generally favored in QCA-only studies, we identify important challenges that can emerge when selecting those solutions in a QCA-PT multimethod study. We particularly highlight the risk of mechanistic heterogeneity, omitted conditions, and draw the attention on the issue of generalization. We discuss each of these intertwined challenges in depth, and explain why the conservative solution is useful to consider in addressing them. We substantiate our arguments by drawing on a recently completed evaluation study that was commissioned by the Flemish ESF Agency in Belgium. In the study, we combined QCA and theory-guided in-depth process-tracing to uncover under what combinations of conditions (QCA) a training programme would lead to successful training transfer and how (PT) this happened in the successful cases. The article highlights the need to carefully consider the selection of solution types in any multimethod design comprising QCA.The politics and administration of institutional chang

Aerobic interval training and continuous training equally improve aerobic exercise capacity in patients with coronary artery disease:The SAINTEX-CAD study

AbstractBackgroundExercise-based cardiac rehabilitation increases peak oxygen uptake (peak VO2), which is an important predictor of mortality in cardiac patients. However, it remains unclear which exercise characteristics are most effective for improving peak VO2 in coronary artery disease (CAD) patients. Proof of concept papers comparing Aerobic Interval Training (AIT) and Moderate Continuous Training (MCT) were conducted in small sample sizes and findings were inconsistent and heterogeneous. Therefore, we aimed to compare the effects of AIT and Aerobic Continuous Training (ACT) on peak VO2, peripheral endothelial function, cardiovascular risk factors, quality of life and safety, in a large multicentre study.MethodsTwo-hundred CAD patients (LVEF >40%, 90% men, mean age 58.4±9.1years) were randomized to a supervised 12-week cardiac rehabilitation programme of three weekly sessions of either AIT (90–95% of peak heart rate (HR)) or ACT (70–75% of peak HR) on a bicycle. Primary outcome was peak VO2; secondary outcomes were peripheral endothelial function, cardiovascular risk factors, quality of life and safety.ResultsPeak VO2 (ml/kg/min) increased significantly in both groups (AIT 22.7±17.6% versus ACT 20.3±15.3%; p-time<0.001). In addition, flow-mediated dilation (AIT+34.1% (range –69.8 to 646%) versus ACT+7.14% (range –66.7 to 503%); p-time<0.001) quality of life and some other cardiovascular risk factors including resting diastolic blood pressure and HDL-C improved significantly after training. Improvements were equal for both training interventions.ConclusionsContrary to earlier smaller trials, we observed similar improvements in exercise capacity and peripheral endothelial function following AIT and ACT in a large population of CAD patients

Disentangling the drivers of future Antarctic ice loss with a historically calibrated ice-sheet model

We use an observationally calibrated ice-sheet model to investigate the future trajectory of the Antarctic ice sheet related to uncertainties in the future balance between sub-shelf melting and ice discharge, on the one hand, and the surface mass balance, on the other. Our ensemble of simulations, forced by a panel of climate models from the sixth phase of the Coupled Model Intercomparison Project (CMIP6), suggests that the ocean will be the primary driver of short-term Antarctic mass loss, initiating ice loss in West Antarctica already during this century. The atmosphere initially plays a mitigating role through increased snowfall, leading to an Antarctic contribution to global mean sea-level rise by 2100 of 6 (−8 to 15) cm under a low-emission scenario and 5.5 (−10 to 16) cm under a very high-emission scenario. However, under the very high-emission pathway, the influence of the atmosphere shifts beyond the end of the century, becoming an amplifying driver of mass loss as the ice sheet's surface mass balance decreases. We show that this transition occurs when Antarctic near-surface warming exceeds a critical threshold of +7.5 ∘C, at which the increase in surface runoff outweighs the increase in snow accumulation, a signal that is amplified by the melt–elevation feedback. Therefore, under the very high-emission scenario, oceanic and atmospheric drivers are projected to result in a complete collapse of the West Antarctic ice sheet along with significant grounding-line retreat in the marine basins of the East Antarctic ice sheet, leading to a median global mean sea-level rise of 2.75 (6.95) m by 2300 (3000). Under a more sustainable socio-economic pathway, we find that the Antarctic ice sheet may still contribute to a median global mean sea-level rise of 0.62 (1.85) m by 2300 (3000). However, the rate of sea-level rise is significantly reduced as mass loss is likely to remain confined to the Amundsen Sea Embayment, where present-day climate conditions seem sufficient to commit to a continuous retreat of Thwaites Glacier.</p