Reflections from a Realist Evaluation in Progress: Scaling Ladders and Stitching Theory

Realist evaluation provides valuable insights into how and why programmes lead to change, and can generate transferable lessons to help practitioners roll out or scale up an intervention. However, as yet there are few standards and guidelines governing what counts as a ‘good’ realist evaluation. This CDI Practice Paper, written by Melanie Punton, Isabel Vogel and Rob Lloyd, reflects on the first year of a three-year realist impact evaluation, examining the Building Capacity to Use Research Evidence (BCURE) programme funded by the UK Department for International Development (DFID). It describes some of the challenges faced and lessons learned, providing insights into the potential value of realist approaches within international development.UK Department for International Developmen

Policymakers\u27 experience of a capacity-building intervention designed to increase their use of research: A realist process evaluation

Background: An intervention’s success depends on how participants interact with it in local settings. Process evaluation examines these interactions, indicating why an intervention was or was not effective, and how it (and similar interventions) can be improved for better contextual fit. This is particularly important for innovative trials like Supporting Policy In health with Research: an Intervention Trial (SPIRIT), where causal mechanisms are poorly understood. SPIRIT was testing a multi-component intervention designed to increase the capacity of health policymakers to use research. Methods: Our mixed-methods process evaluation sought to explain variation in observed process effects across the six agencies that participated in SPIRIT. Data collection included observations of intervention workshops (n = 59), purposively sampled interviews (n = 76) and participant feedback forms (n = 553). Using a realist approach, data was coded for context-mechanism-process effect configurations (retroductive analysis) by two authors. Results: Intervention workshops were very well received. There was greater variation of views regarding other aspects of SPIRIT such as data collection, communication and the intervention’s overall value. We identified nine inter-related mechanisms that were crucial for engaging participants in these policy settings: (1) Accepting the premise (agreeing with the study’s assumptions); (2) Self-determination (participative choice); (3) The Value Proposition (seeing potential gain); (4) ‘Getting good stuff’ (identifying useful ideas, resources or connections); (5) Self-efficacy (believing ‘we can do this!’); (6) Respect (feeling that SPIRIT understands and values one’s work); (7) Confidence (believing in the study’s integrity and validity); (8) Persuasive leadership (authentic and compelling advocacy from leaders); and (9) Strategic insider facilitation (local translation and mediation). These findings were used to develop tentative explanatory propositions and to revise the programme theory. Conclusion: This paper describes how SPIRIT functioned in six policy agencies, including why strategies that worked well in one site were less effective in others. Findings indicate a complex interaction between participants’ perception of the intervention, shifting contextual factors, and the form that the intervention took in each site. Our propositions provide transferable lessons about contextualised areas of strength and weakness that may be useful in the development and implementation of similar studies

Developing programme theories as part of a realist evaluation of a healthcare quality improvement programme

The application of realist approaches to health services research and evaluation has continued to grow over the past two decades. However, difficulties in defining and operationalising key realist concepts of contexts, mechanisms and outcomes in healthcare settings continue to be recognised within the realist evaluation literature. Reflecting on an ongoing realist evaluation of a healthcare quality improvement programme across an inter-organisational context, this article explores some of the methodological challenges encountered by the author in the early stages of programme theory development. An individualised operationalisation and application of realist concepts are presented to demonstrate how initial programme theories can be developed despite the methodological difficulties presented.</p

Evaluating the adolescents 360 approach in Ogun (Nigeria), Nasarawa (Nigeria), Oromia (Ethiopia) and Mwanza (Tanzania) – Study data

Adolescents 360 (A360) was an approach in the field of adolescent sexual and reproductive health programming, with the intention of being implemented at scale in Nigeria, Ethiopia and Tanzania. We present here the data for the outcome evaluation of A360. The primary objective of the outcome evaluation was to evaluate the effectiveness of the A360 programme in increasing the uptake of voluntary modern contraception among sexually active girls aged 15–19 years