Citizens and Evaluation: A Review of Evaluation Models

Evaluations are considered of key importance for a well-functioning democracy. Against this background, it is vital to assess whether and how evaluation models approach the role of citizens. This paper is the first in presenting a review of citizen involvement in the main evaluation models which are commonly distinguished in the field. We present the results of both a document analysis and an international survey with experts who had a prominent role in developing the models. This overview has not only a theoretical relevance, but can also be helpful for evaluation practitioners or scholars looking for opportunities for citizen involvement. The paper contributes to the evaluation literature in the first place, but also aims to fine-tune available insights on the relationship between evidence informed policy making and citizens

Political and social forces shaping political science research and knowledge transfer in the Netherlands

The authors identify a third type of relevance that sits between the scholar-selected and state-directed poles that were identified in the opening chapter. This "middle position" is that of the pragmatic scholar seeking windows for narrowing funding possibilities of fundamental research, at the same time engaging freely with the potential research users that are the targets of impact. This type, referred to as "entrepreneurial relevance, seems to represent some of the key features of the Dutch context-such as the country's pragmatic and consensus-building political and social culture-, and it also matches findings on the advisory roles of political scientists".</p

Qualitative comparative analysis: Meer behorend bij de kwalitatieve dan de kwantitatieve benadering

Qualitative Comparative Analysis (QCA) is an approach that enables systematically comparing qualitative case study data. The fact that QCA borrows certain elements from quantitative methods (such as the use of Boolean algebra and the quantification of conditions and outcomes) may raise questions about the qualitative character of QCA. Based on a discussion of four criteria relating to causal analysis, we argue that QCA belongs to the qualitative research tradition. Where QCA borrows characteristics from the quantitative approach, this is only for the sake of a qualitative objective: to better understand the conditions underlying a particular outcome or phenomenon

The monitoring and evaluation challenges of the sustainable development goals : an assessment in three European countries

While discourse about the Sustainable Development Goals (SDGs) has primarily focused on ‘whether the goals’ are achieved, there remains limited understanding of how developed countries organize their monitoring and evaluation (M&E) systems, which play a crucial role in tracking progress towards the SDGs. In this contribution, we unpack the M&E frameworks of Belgium, the Netherlands and the United Kingdom. To do so, we have devised an analytical heuristic that combines insights from the literature on policy performance measurement and measurement infrastructures with the more specific literature on SDG governance. Through document analysis and elite interviews conducted in 2021, our findings highlight similarity in underdeveloped M&E frameworks, rather than significant variation across the three case studies. The results do not suggest a linkage between SDG performance and the development of M&E frameworks

Focal DNA copy number changes in neuroblastoma target MYCN regulated genes

Neuroblastoma is an embryonic tumor arising from immature sympathetic nervous system cells. Recurrent genomic alterations include MYCN and ALK amplification as well as recurrent patterns of gains and losses of whole or large partial chromosome segments. A recent whole genome sequencing effort yielded no frequently recurring mutations in genes other than those affecting ALK. However, the study further stresses the importance of DNA copy number alterations in this disease, in particular for genes implicated in neuritogenesis. Here we provide additional evidence for the importance of focal DNA copy number gains and losses, which are predominantly observed in MYCN amplified tumors. A focal 5 kb gain encompassing the MYCN regulated miR-17,92 cluster as sole gene was detected in a neuroblastoma cell line and further analyses of the array CGH data set demonstrated enrichment for other MYCN target genes in focal gains and amplifications. Next we applied an integrated genomics analysis to prioritize MYCN down regulated genes mediated by MYCN driven miRNAs within regions of focal heterozygous or homozygous deletion. We identified RGS5, a negative regulator of G-protein signaling implicated in vascular normalization, invasion and metastasis, targeted by a focal homozygous deletion, as a new MYCN target gene, down regulated through MYCN activated miRNAs. In addition, we expand the miR-17,92 regulatory network controlling TGFß signaling in neuroblastoma with the ring finger protein 11 encoding gene RNF11, which was previously shown to be targeted by the miR-17,92 member miR-19b. Taken together, our data indicate that focal DNA copy number imbalances in neuroblastoma (1) target genes that are implicated in MYCN signaling, possibly selected to reinforce MYCN oncogene addiction and (2) serve as a resource for identifying new molecular targets for treatment

Taking Care of Policy in Times of Crisis: Comparative Lessons from Belgium’s Longest Caretaker Government

When the financial crisis hit the Eurozone, Belgium, along with several other countries, postponed its consolidation policies until after the general elections. What followed was the longest caretaker rule that any stable democracy had ever experienced. This article analyses the phenomenon of policy continuity and change during this double crisis. It illustrates how the exogenous economic crisis overrode the endogenous political crisis, showing the extent to which international policy determinants expanded the remit of caretaker policy-making. At the same time, our analysis of the nature of caretaker conventions, the nature of multi-level governance, the permanence of administrative personnel, and the re-invention of parliament offers opportunities to draw lessons and deepen comparative research on policy termination and maintenance in the face of crisis.status: publishe

Why organisations (do not) evaluate: A search for necessary and sufficient conditions

The wide acceptance of evaluation in this evidence-based society might hide still significant variation in the extent of evaluation activeness between public sector organisations. In explaining these differences, evidence is only fragmentally available. Admittedly, multiple explanatory factors can be identified in the evaluation community, mainly in the evaluation capacity building literature. Yet, common to the practical character of the field, insights are mainly of anecdotic nature and have seldom been systematically tested. Thus far, the only certainty is that ‘contingency’ matters. The inherent contingent nature of evaluation practices may not discourage us, however, from collecting more systematic insight in explaining differences in the extent of evaluation activeness. It is not clear, indeed, to which extent the contingency reigns. The question is whether more parsimonious patterns can nonetheless be discerned, when attacking the complexity. The present paper takes up this challenge. Via a systematic comparison of 27 public sector organisations of the Flemish administration (Belgium) through the application of several configurational comparative methods, the analysis identifies a range of necessary and sufficient (combinations of) conditions for the (non)conduct of evaluations.status: publishe

Why organisations (do not) evaluate? Explaining evaluation activity through the lens of configurational comparative methods

© The Author(s) 2014. This article aims at explaining why some Flemish (Belgian) organizations evaluate policy, while others do not. The study relies on a unique combination of two configurational comparative methods: the Most Similar Different Outcome/Most Different Similar Outcome method and crisp set Qualitative Comparative Analysis. This combination of methods helps us unravel the combinations of conditions that promote or impede policy evaluation activity in a public administration that recently underwent major changes in line with New Public Management (NPM). The analysis reveals that the impact of NPM reforms on evaluation activity should not be overestimated. The results suggest that the important conditions that explain policy evaluation activity or inactivity are the anchorage of the evaluation function, the availability of skilled personnel to undertake evaluations, evaluation demand from organizational management, and the measurability of the outputs and outcomes of the organization’s activities.status: publishe

Why organizations (do not) evaluate: a search for necessary and sufficient conditions

The wide acceptance of evaluation in this evidence-based society might hide significant variation in the extent of evaluation activeness between public sector organizations. In explaining these differences, evidence is only fragmentally available. Admittedly, multiple explanatory factors can be identified in the evaluation community, mainly in the evaluation capacity building literature. Yet, common to the practical character of the field, insights are mainly of anecdotic nature and have seldom been systematically tested. Thus far, the only certainty is that ‘contingency’ matters. The inherently contingent nature of evaluation practices may not discourage us, however, from collecting more systematic insight in explaining differences in the extent of evaluation activeness. It is not clear, indeed, to which degree the contingency reigns. The question is whether more parsimonious patterns can nonetheless be discerned, when attacking the complexity. The present paper takes up this challenge. Via a systematic comparison of 27 public sector organizations of the Flemish administration (Belgium) through the application of several configurational comparative techniques (MSDO/MDSO & csQCA), the analysis identifies a range of necessary and sufficient (combinations of) conditions for the (non)conduct of evaluations.No ISSNstatus: publishe

Contingency questioned. Understanding the dynamics of policy evaluation practices in public sector organisations

Policy evaluation practice, as an applied policy analysis tool, has in general gained considerable terrain in the public arena, in Europe and abroad. Notwithstanding this trend, it does not seem appropriate to talk in terms of “a” European evaluation tradition, or “a” European evaluation culture. Mapping different countries’ evaluation traditions, would reveal an ‘evaluation tree with different trunks and branches’. Both the trajectories (‘roots’) of its development, as well as its resulting evaluation discourse, practices and institutions, largely differ throughout Europe. In explaining the similarities/differences in evaluation traditions an almost general consensus exists on the importance of societal, legal, institutional and historical contexts. Yet, systematic research evidence on this phenomenon is generally lacking. The only certainty remaining is the apparent highly contingent nature of evaluation practices. Yet, its inherent contingency might not discourage more systematic research to understand “why evaluation practices look like they do in the public sector”, and to get more grips on the role of various contextual factors herein. Remarkably, this is an area in which systematic research about evaluation is almost non-existent. Moreover, it is striking to notice that the majority of available analytical schemes concentrate at a single level of analysis (biased towards the country level) without taking into account possible interaction with other levels (e.g. sectoral; organisational). Conducting systematic multilevel research in search of possible combinations of contextual factors clarifying evaluation practices would be of high relevance. Not only for mere theoretical purposes, but also to indicate possible levers for evaluation capacity building. The paper presents a (PhD) research design, set up to tackle (part of) this research agenda. It approaches the general label ‘evaluation practices’ from two angles: first, from a developmental perspective. The ambition here is to explain the maturity of the evaluation capacity of a public sector organisation on a “plan-do-check-act” management scale. Secondly, evaluation practices are studied from an evaluation cycle outlook. We focus on explaining the loci of evaluation initiation, implementation and the number of evaluations undertaken by public sector organisations. Variables are inductively generated and ordered by an actor centered institutionalist framework into actor; institution; policy issue related conditions and evaluation policy legacies. 32 organisations of the Flemish administration constitute the area of investigation. In accordance with the contextual nature of evaluation practices and the theoretical framework applied, the methodology considered most suitable for the analysis is quali-quantitative comparative analysis (QCA). This involves a particular understanding of causality, known as “multiple conjunctural causation”.status: publishe