19 research outputs found
Reflections from a Realist Evaluation in Progress: Scaling Ladders and Stitching Theory
Realist evaluation provides valuable insights into how and why programmes lead to change,
and can generate transferable lessons to help practitioners roll out or scale up an intervention.
However, as yet there are few standards and guidelines governing what counts as a âgoodâ realist evaluation. This CDI Practice Paper, written by Melanie Punton, Isabel Vogel and Rob Lloyd, reflects on the first year of a three-year realist impact evaluation, examining the Building Capacity to Use Research Evidence (BCURE) programme funded by the UK Department for International Development (DFID). It describes some of the challenges faced and lessons learned, providing insights into the potential value of realist approaches within international development.UK Department for International Developmen
Policymakers\u27 experience of a capacity-building intervention designed to increase their use of research: A realist process evaluation
Background: An interventionâs success depends on how participants interact with it in local settings. Process evaluation examines these interactions, indicating why an intervention was or was not effective, and how it (and similar interventions) can be improved for better contextual fit. This is particularly important for innovative trials like Supporting Policy In health with Research: an Intervention Trial (SPIRIT), where causal mechanisms are poorly understood. SPIRIT was testing a multi-component intervention designed to increase the capacity of health policymakers to use research.
Methods: Our mixed-methods process evaluation sought to explain variation in observed process effects across the six agencies that participated in SPIRIT. Data collection included observations of intervention workshops (n = 59), purposively sampled interviews (n = 76) and participant feedback forms (n = 553). Using a realist approach, data was coded for context-mechanism-process effect configurations (retroductive analysis) by two authors.
Results: Intervention workshops were very well received. There was greater variation of views regarding other aspects of SPIRIT such as data collection, communication and the interventionâs overall value. We identified nine inter-related mechanisms that were crucial for engaging participants in these policy settings: (1) Accepting the premise (agreeing with the studyâs assumptions); (2) Self-determination (participative choice); (3) The Value
Proposition (seeing potential gain); (4) âGetting good stuffâ (identifying useful ideas, resources or connections); (5) Self-efficacy (believing âwe can do this!â); (6) Respect (feeling that SPIRIT understands and values oneâs work); (7) Confidence (believing in the studyâs integrity and validity); (8) Persuasive leadership (authentic and compelling advocacy from leaders); and (9) Strategic insider facilitation (local translation and mediation). These findings were used to develop tentative explanatory propositions and to revise the programme theory.
Conclusion: This paper describes how SPIRIT functioned in six policy agencies, including why strategies that worked well in one site were less effective in others. Findings indicate a complex interaction between participantsâ perception of the intervention, shifting contextual factors, and the form that the intervention took in each site. Our propositions provide transferable lessons about contextualised areas of strength and weakness that may be useful in the development and implementation of similar studies
Developing programme theories as part of a realist evaluation of a healthcare quality improvement programme
The application of realist approaches to health services research and evaluation has continued to grow over the past two decades. However, difficulties in defining and operationalising key realist concepts of contexts, mechanisms and outcomes in healthcare settings continue to be recognised within the realist evaluation literature. Reflecting on an ongoing realist evaluation of a healthcare quality improvement programme across an inter-organisational context, this article explores some of the methodological challenges encountered by the author in the early stages of programme theory development. An individualised operationalisation and application of realist concepts are presented to demonstrate how initial programme theories can be developed despite the methodological difficulties presented.</p
What historiography can tell us about evidence-based medicine and policy: A response to Pawsonâs âThe Realist Foundations of Evidence-based Medicineâ
Finding the Impact: Methods for Assessing the Contribution of Collective Impact to Systems and Population Change in a MultiâSite Study
Using computer assisted qualitative data analysis software (CAQDAS; NVivo) to assist in the complex process of realist theory generation, refinement and testing
Reflections on mentoring experiences for evidence-informed decision-making in South Africa and Malawi
Evaluating the adolescents 360 approach in Ogun (Nigeria), Nasarawa (Nigeria), Oromia (Ethiopia) and Mwanza (Tanzania) â Study data
Adolescents 360 (A360) was an approach in the field of adolescent sexual and reproductive health programming, with the intention of being implemented at scale in Nigeria, Ethiopia and Tanzania. We present here the data for the outcome evaluation of A360. The primary objective of the outcome evaluation was to evaluate the effectiveness of the A360 programme in increasing the uptake of voluntary modern contraception among sexually active girls aged 15â19 years