RAMESES II reporting standards for realist evaluations

Realist evaluation is increasingly used in health services and other fields of research and evaluation. No previous standards exist for reporting realist evaluations. This standard was developed as part of the RAMESES II project. The project's aim is to produce initial reporting standards for realist evaluations.We purposively recruited a maximum variety sample of an international group of experts in realist evaluation to our online Delphi panel. Panel members came from a variety of disciplines, sectors and policy fields. We prepared the briefing materials for our Delphi panel by summarising the most recent literature on realist evaluations to identify how and why rigour had been demonstrated and where gaps in expertise and rigour were evident. We also drew on our collective experience as realist evaluators, in training and supporting realist evaluations, and on the RAMESES email list to help us develop the briefing materials. Through discussion within the project team, we developed a list of issues related to quality that needed to be addressed when carrying out realist evaluations. These were then shared with the panel members and their feedback was sought. Once the panel members had provided their feedback on our briefing materials, we constructed a set of items for potential inclusion in the reporting standards and circulated these online to panel members. Panel members were asked to rank each potential item twice on a 7-point Likert scale, once for relevance and once for validity. They were also encouraged to provide free text comments.We recruited 35 panel members from 27 organisations across six countries from nine different disciplines. Within three rounds our Delphi panel was able to reach consensus on 20 items that should be included in the reporting standards for realist evaluations. The overall response rates for all items for rounds 1, 2 and 3 were 94 %, 76 % and 80 %, respectively.These reporting standards for realist evaluations have been developed by drawing on a range of sources. We hope that these standards will lead to greater consistency and rigour of reporting and make realist evaluation reports more accessible, usable and helpful to different stakeholders

Care Staff and the Creative Arts: Exploring the Context of Involving Care Personnel in Arts Interventions

Background: Arts-based interventions play an important role in the care of people with dementia. Yet, creative arts are seldom implemented as a tool to enhance the care and wellbeing of people with dementia. Methods: We examined the involvement of care staff in creative arts activities in residential care. Aspects of involvement that appear to influence outcomes in people with dementia were identified and analyzed. A broad systematic literature search of MedLine, EMBASE, PsychInfo, CINAHL, ASSIA, SCOPUS, and Web of Science led to the identification of 14 papers. The studies identified through the search process were examined in terms of intervention, context, mechanism and outcome, and the relationships between these aspects. Results: Training sessions were identified as an opportunity to educate care personnel on useful techniques that are relevant to daily care practice. Evidence from the literature suggests that creative arts programs play a significant role in the way staff and residents interact and as a result influence the care practice of staff. Under certain conditions creative arts programs, that involve and engage staff, facilitate enhanced interactions and improve care strategies, which leads to the recognition and validation of personhood in residents with dementia. Conclusions: These findings provide a basis for illustrating which elements of care staff involvement in creative arts programs could be implemented in residential care contexts in order to have the upmost benefit. Copyright © International Psychogeriatric Association 2017

Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review

Objective: There is a widely held assumption that engagement by clinicians and healthcare organisations in research improves healthcare performance at various levels, but little direct empirical evidence has previously been collated. The objective of this study was to address the question: Does research engagement (by clinicians and organizations) improve healthcare performance? Methods: An hourglass-shaped review was developed, consisting of three stages: (1) a planning and mapping stage; (2) a focused review concentrating on the core question of whether or not research engagement improves healthcare performance; and (3) a wider (but less systematic) review of papers identified during the two earlier stages, focusing on mechanisms. Results: Of the 33 papers included in the focused review, 28 identified improvements in health services performance. Seven out of these papers reported some improvement in health outcomes, with others reporting improved processes of care. The wider review demonstrated that mechanisms such as collaborative and action research can encourage some progress along the pathway from research engagement towards improved healthcare performance. Organisations that have deliberately integrated the research function into organisational structures demonstrate how research engagement can, among other factors, contribute to improved healthcare performance. Conclusions: Current evidence suggests that there is an association between the engagement of individuals and healthcare organisations in research and improvements in healthcare performance. The mechanisms through which research engagement might improve healthcare performance overlap and rarely act in isolation, and their effectiveness often depends on the context in which they operate. Strengths and limitations of this study • This review brings together for the first time a diverse body of literature addressing whether engaging clinicians and healthcare organisations in research is the likely to improve healthcare performance • It also explores the mechanisms through which improvement is achieved to try and understand how any improvements might come about • However, it relies on the quality and coverage of the existing literature • It is an extremely complex topic, but nonetheless one worthy of further exploration, particularly given the pressure to justify research spending in healthcare systems, and to encourage its implementation.UK National Institute for Health Research (NIHR) Service Delivery and Organisation (SDO) programme ( project number HS&DR- 10/1012/09

Protocol—the RAMESES II study:developing guidance and reporting standards for realist evaluation

Introduction: Realist evaluation is an increasingly popular methodology in health services research. For realist evaluations (RE) this project aims to: develop quality and reporting standards and training materials; build capacity for undertaking and critically evaluating them; produce resources and training materials for lay participants, and those seeking to involve them. Methods: To achieve our aims, we will: (1) Establish management and governance infrastructure; (2) Recruit an interdisciplinary Delphi panel of 35 participants with diverse relevant experience of RE; (3) Summarise current literature and expert opinion on best practice in RE; (4) Run an online Delphi panel to generate and refine items for quality and reporting standards; (5) Capture ‘real world’ experiences and challenges of RE—for example, by providing ongoing support to realist evaluations, hosting the RAMESES JISCmail list on realist research, and feeding problems and insights from these into the deliberations of the Delphi panel; (6) Produce quality and reporting standards; (7) Collate examples of the learning and training needs of researchers, students, reviewers and lay members in relation to RE; (8) Develop, deliver and evaluate training materials for RE and deliver training workshops; and (9) Develop and evaluate information and resources for patients and other lay participants in RE (eg, draft template information sheets and model consent forms) and; (10) Disseminate training materials and other resources. Planned outputs: (1) Quality and reporting standards and training materials for RE. (2) Methodological support for RE. (3) Increase in capacity to support and evaluate RE. (4) Accessible, plain-English resources for patients and the public participating in RE. Discussion: The realist evaluation is a relatively new approach to evaluation and its overall place in the is not yet fully established. As with all primary research approaches, guidance on quality assurance and uniform reporting is an important step towards improving quality and consistenc

RAMESES publication standards: realist syntheses

PMCID: PMC3558331This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited

Adapting realist synthesis methodology: The case of workplace harassment interventions

Realist synthesis techniques can be used to assess complex interventions by extracting and synthesizing configurations of contexts, mechanisms, and outcomes found in the literature. Our novel and multi‐pronged approach to the realist synthesis of workplace harassment interventions describes our pursuit of theory to link macro and program level theories. After discovering the limitations of a dogmatic approach to realist synthesis, we adapted our search strategy and focused our analysis on a subset of data. We tailored our realist synthesis to understand how, why, and under what circumstances workplace harassment interventions are effective. The result was a conceptual framework to test our theory‐based interventions and provide the basis for subsequent realist evaluation. Our experience documented in this article contributes to an understanding of how, under what circumstances, and with what consequences realist synthesis principles can be customized

Interventions to improve antimicrobial prescribing of doctors in training: the IMPACT (IMProving Antimicrobial presCribing of doctors in Training) realist review

Journal ArticleINTRODUCTION: Antimicrobial resistance has been described as a global crisis-more prudent prescribing is part of the solution. Behaviour change interventions are needed to improve prescribing practice. Presently, the literature documents that context impacts on prescribing decisions, yet insufficient evidence exists to enable researchers and policymakers to determine how local tailoring should take place. Doctors in training are an important group to study, being numerically the largest group of prescribers in UK hospitals. Unfortunately very few interventions specifically targeted this group. METHODS AND ANALYSIS: Our project aims to understand how interventions to change antimicrobial prescribing behaviours of doctors in training produce their effects. We will recruit a project stakeholder group to advise us throughout. We will synthesise the literature using the realist review approach-a form of theory-driven interpretive systematic review approach often used to make sense of complex interventions. Interventions to improve antimicrobial prescribing behaviours are complex-they are context dependent, have long implementation chains, multiple non-linear interactions, emergence and depend on human agency. Our review will iteratively progress through 5 steps: step 1-Locate existing theories; step 2-Search for evidence; step 3-Article selection; step 4-Extracting and organising data; and step 5-Synthesising the evidence and drawing conclusions. Data analysis will use a realist logic of analysis to describe and explain what works, for whom, in what circumstances, in what respects, how and why to improve antimicrobial prescribing behaviour of doctors in training. ETHICS AND DISSEMINATION: Ethical approval was not required for our review. Our dissemination strategy will be participatory and involve input from our stakeholder group. Tailored project outputs will be targeted at 3 audiences: (1) doctors in training; (2) clinical supervisors/trainers and medical educators; and (3) policy, decision makers, regulators and royal societies.NIHRNIHR / CLAHR

ReseArch with Patient and Public invOlvement: a RealisT evaluation - the RAPPORT study

Background Patient and public involvement (PPI) is a prerequisite for many funding bodies and NHS research ethics approval. PPI in research is defined as research carried out with or by the public rather than to, about or for them. While the benefits of PPI have been widely discussed, there is a lack of evidence on the impact and outcomes of PPI in research. Objectives To determine the types of PPI in funded research, describe key processes, analyse the contextual and temporal dynamics of PPI and explore the experience of PPI in research for all those involved. Mechanisms contributing to the routine incorporation of PPI in the research process were assessed, the impact of PPI on research processes and outcomes evaluated, and barriers and enablers to effective PPI identified. Design A three-staged realist evaluation drawing on Normalisation Process Theory to understand how far PPI was embedded within health-care research in six areas: diabetes mellitus, arthritis, cystic fibrosis, dementia, public health and learning disabilities. The first two stages comprised a scoping exercise and online survey to chief investigators to assess current PPI activity. The third stage consisted of case studies tracked over 18 months through interviews and document analysis. The research was conducted in four regions of England. Participants Non-commercial studies currently running or completed within the previous 2 years eligible for adoption on the UK Clinical Research Network portfolio. A total of 129 case study participants included researchers and PPI representatives from 22 research studies, and representatives from funding bodies and PPI networks