RAMESES publication standards: realist syntheses

PMCID: PMC3558331This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited

Developing and testing intervention theory by incorporating a views synthesis into a qualitative comparative analysis of intervention effectiveness.

Qualitative comparative analysis (QCA) was originally developed as a tool for cross-national comparisons in macrosociology, but its use in evaluation and evidence synthesis of complex interventions is rapidly developing. QCA is theory-driven and relies on Boolean logic to identify pathways to an outcome (eg, is the intervention effective or not?). We use the example of two linked systematic reviews on weight management programs (WMPs) for adults-one focusing on user views (a "views synthesis") and one focusing on the effectiveness of WMPs incorporating dietary and physical activity-to demonstrate how a synthesis of user views can supply a working theory to structure a QCA. We discuss how a views synthesis is especially apt to supply this working theory because user views can (a) represent a "middle-range theory" of the intervention; (b) bring a participatory, democratic perspective; and (c) provide an idiographic understanding of how the intervention works that external taxonomies may not be able to furnish. We then discuss the practical role that the views synthesis played in our QCA examining pathways to effectiveness: (a) by suggesting specific intervention features and sharpening the focus on the most salient features to be examined, (b) by supporting interpretation of findings, and (c) by bounding data analysis to prevent data dredging

Using non-participant observation to uncover mechanisms: insights from a realist evaluation

This article outlines how a realist evaluation of dementia care in hospitals used non-participant observation to support the refinement and testing of mechanisms likely to lead to the use of person-centred care. We found that comments and explanations of their actions from hospital staff during observation periods provided insights into the reasoning that generated their actions for care in real time. This informed subsequent data collection and analysis. Two worked examples of mechanisms first identified during non-participant observation demonstrate (1) how they were uncovered, and (2) how this informed research activities for theory refinement. Early, iterative engagement with the analytic process, primarily involving reflection and debate with the research team, maximised the potential of observation data to support surfacing underlying mechanisms, linking them to specific contexts and outcomes.Peer reviewedFinal Accepted Versio

Introducing EMMIE: An evidence rating scale to encourage mixed-method crime prevention synthesis reviews

Objectives This short report describes the need for, and the development of, a coding system to distil the quality and coverage of systematic reviews of the evidence relating to crime prevention interventions. The starting point for the coding system concerns the evidence needs of policymakers and practitioners. Methods The coding scheme (EMMIE) proposed builds on previous scales that have been developed to assess the probity, coverage and utility of evidence both in health and criminal justice. It also draws on the principles of realist synthesis and review. Results The proposed EMMIE scale identifies five dimensions to which systematic reviews intended to inform crime prevention should speak. These are the Effect of intervention, the identification of the causal Mechanism(s) through which interventions are intended to work, the factors that Moderate their impact, the articulation of practical Implementation issues, and the Economic costs of intervention

Scoping studies: towards a methodological framework

This paper focuses on scoping studies, an approach to reviewing the literature which to date has received little attention in the research methods literature. We distinguish between different types of scoping studies and indicate where these stand in relation to full systematic reviews. We outline a framework for conducting a scoping study based on our recent experiences of reviewing the literature on services for carers for people with mental health problems. Where appropriate, our approach to scoping the field is contrasted with the procedures followed in systematic reviews. We emphasize how including a consultation exercise in this sort of study may enhance the results, making them more useful to policy makers, practitioners and service users. Finally, we consider the advantages and limitations of the approach and suggest that a wider debate is called for about the role of the scoping study in relation to other types of literature reviews

Enterprise Education Competitions: A Theoretically Flawed Intervention?

The demand for including enterprise in the education system, at all levels and for all pupils is now a global phenomenon. Within this context, the use of competitions and competitive learning activities is presented as a popular and effective vehicle for learning. The purpose of this chapter is to illustrate how a realist method of enquiry – which utilises theory as the unit of analysis – can shed new light on the assumed and unintended outcomes of enterprise education competitions. The case developed here is that there are inherent flaws in assuming that competitions will ‘work’ in the ways set out in policy and guidance. Some of the most prevalent stated outcomes – that competitions will motivate and reward young people, that they will enable the development of entrepreneurial skills, and that learners will be inspired by their peers – are challenged by theory from psychology and education. The issue at stake is that the expansion of enterprise education policy into primary and secondary education increases the likelihood that more learners will be sheep dipped in competitions, and competitive activities, without a clear recognition of the potential unintended effects. In this chapter, we employ a realist-informed approach to critically evaluate the theoretical basis that underpins the use of competitions and competitive learning activities in school-based enterprise education. We believe that our findings and subsequent recommendations will provide those who promote and practice the use of competitions with a richer, more sophisticated picture of the potential flaws within such activities.Peer reviewedFinal Published versio

A realistic evaluation : the case of protocol-based care

Background 'Protocol based care' was envisioned by policy makers as a mechanism for delivering on the service improvement agenda in England. Realistic evaluation is an increasingly popular approach, but few published examples exist, particularly in implementation research. To fill this gap, within this paper we describe the application of a realistic evaluation approach to the study of protocol-based care, whilst sharing findings of relevance about standardising care through the use of protocols, guidelines, and pathways. Methods Situated between positivism and relativism, realistic evaluation is concerned with the identification of underlying causal mechanisms, how they work, and under what conditions. Fundamentally it focuses attention on finding out what works, for whom, how, and in what circumstances. Results In this research, we were interested in understanding the relationships between the type and nature of particular approaches to protocol-based care (mechanisms), within different clinical settings (context), and what impacts this resulted in (outcomes). An evidence review using the principles of realist synthesis resulted in a number of propositions, i.e., context, mechanism, and outcome threads (CMOs). These propositions were then 'tested' through multiple case studies, using multiple methods including non-participant observation, interviews, and document analysis through an iterative analysis process. The initial propositions (conjectured CMOs) only partially corresponded to the findings that emerged during analysis. From the iterative analysis process of scrutinising mechanisms, context, and outcomes we were able to draw out some theoretically generalisable features about what works, for whom, how, and what circumstances in relation to the use of standardised care approaches (refined CMOs). Conclusions As one of the first studies to apply realistic evaluation in implementation research, it was a good fit, particularly given the growing emphasis on understanding how context influences evidence-based practice. The strengths and limitations of the approach are considered, including how to operationalise it and some of the challenges. This approach provided a useful interpretive framework with which to make sense of the multiple factors that were simultaneously at play and being observed through various data sources, and for developing explanatory theory about using standardised care approaches in practice

Fluvial organic carbon flux from an eroding peatland catchment, southern Pennines, UK

International audienceThis study investigates for the first time the relative importance of dissolved organic carbon (DOC) and particulate organic carbon (POC) in the fluvial carbon flux from an actively eroding peatland catchment in the southern Pennines, UK. Event scale variability in DOC and POC was examined and the annual flux of fluvial organic carbon was estimated for the catchment. At the event scale, both DOC and POC were found to increase with discharge, with event based POC export accounting for 95% of flux in only 8% of the time. On an annual cycle, 40.8 t organic carbon (OC) is exported from the catchment, which represents an areal value of 107 gC m?2 a?1. POC was the most significant form of organic carbon export, accounting for ~82% of the estimated flux. This suggests that more research is required on both the fate of POC and the rates of POC export in eroding peatland catchments

Evaluating the impact of post-qualifying social work education.

Post-qualifying awards in social work are well established within the continuing professional development agenda for qualified social workers in the UK. The evaluation of education and training should be an integral part of this agenda because it is important to ensure that programmes continue to meet standards of delivery, are successful in meeting their aims and objectives and are making an impact on practice. However, there is limited amount of published work on the evaluation of post-qualifying social work education, with studies often focusing on programme delivery rather than on their impact on practice. This paper explores evaluative work within the current post-qualifying social work framework, and discusses the results of an evaluation of the Vulnerable Adults and Community Care Practice programme, a specialist post-qualifying social work education programme run by a UK university, as an example of an evaluation of the impact on practice. The results indicate positive evidence of impact on practice and demonstrate examples of how the programme has had a direct effect on individuals, teams, organisations and on people who use services

Exploring complex causal pathways between urban renewal, health and health inequality using a theory-driven approach

Urban populations are growing and to accommodate these numbers, cities are becoming more involved in urban renewal programs to improve the physical, social and economic conditions in different areas. This paper explores some of the complexities surrounding the link between urban renewal, health and health inequalities using a theory-driven approach. ; We focus on an urban renewal initiative implemented in Barcelona, the Neighbourhoods Law, targeting Barcelona’s (Spain) most deprived neighbourhoods. We present evidence from two studies on the health evaluation of the Neighbourhoods Law, while drawing from recent urban renewal literature, to follow a four-step process to develop a program theory. We then use two specific urban renewal interventions, the construction of a large central plaza and the repair of streets and sidewalks, to further examine this link. ; In order for urban renewal programs to affect health and health inequality, neighbours must use and adapt to the changes produced by the intervention. However, there exist barriers that can result in negative outcomes including factors such as accessibility, safety and security. ; This paper provides a different perspective to the field that is largely dominated by traditional quantitative studies that are not always able to address the complexities such interventions provide. Furthermore, the framework and discussions serve as a guide for future research, policy development and evaluation