Evaluating the successful implementation of evidence into practice using the PARiHS framework : theoretical and practical challenges

Background The PARiHS framework (Promoting Action on Research Implementation in Health Services) has proved to be a useful practical and conceptual heuristic for many researchers and practitioners in framing their research or knowledge translation endeavours. However, as a conceptual framework it still remains untested and therefore its contribution to the overall development and testing of theory in the field of implementation science is largely unquantified. Discussion This being the case, the paper provides an integrated summary of our conceptual and theoretical thinking so far and introduces a typology (derived from social policy analysis) used to distinguish between the terms conceptual framework, theory and model – important definitional and conceptual issues in trying to refine theoretical and methodological approaches to knowledge translation. Secondly, the paper describes the next phase of our work, in particular concentrating on the conceptual thinking and mapping that has led to the generation of the hypothesis that the PARiHS framework is best utilised as a two-stage process: as a preliminary (diagnostic and evaluative) measure of the elements and sub-elements of evidence (E) and context (C), and then using the aggregated data from these measures to determine the most appropriate facilitation method. The exact nature of the intervention is thus determined by the specific actors in the specific context at a specific time and place. In the process of refining this next phase of our work, we have had to consider the wider issues around the use of theories to inform and shape our research activity; the ongoing challenges of developing robust and sensitive measures; facilitation as an intervention for getting research into practice; and finally to note how the current debates around evidence into practice are adopting wider notions that fit innovations more generally. Summary The paper concludes by suggesting that the future direction of the work on the PARiHS framework is to develop a two-stage diagnostic and evaluative approach, where the intervention is shaped and moulded by the information gathered about the specific situation and from participating stakeholders. In order to expedite the generation of new evidence and testing of emerging theories, we suggest the formation of an international research implementation science collaborative that can systematically collect and analyse experiences of using and testing the PARiHS framework and similar conceptual and theoretical approaches. We also recommend further refinement of the definitions around conceptual framework, theory, and model, suggesting a wider discussion that embraces multiple epistemological and ontological perspectives

Lessons learnt from the Bristol Girls Dance Project cluster RCT: Implications for designing and implementing after-school physical activity interventions

Objective: To consider implementation issues associated with the delivery of Bristol Girls Dance Project (BGDP) and to identify improvements that may aid the design of after-school physical activity (PA) interventions. Design: Two-armed cluster randomised control trial. The BGDP was a 20-week school-based intervention, consisting of two 75 min after-school dance sessions per week, which aimed to support Year 7 girls to be more physically active. Setting: 18 secondary schools (nine intervention, nine control) in the Greater Bristol area (as an indication of deprivation, children eligible for the pupil premium in participant schools ranged from 6.9 to 53.3%). Participants: 571 Year 7 girls. This article reports on qualitative data collected from 59 girls in the intervention arm of the trial, 10 dance instructors and 9 school contacts involved in the delivering of the BGDP. Methods: Data were obtained from nine focus groups with girls (one per intervention school), and interviews with dance instructors and school contacts. Focus groups sought views of girls' motivation to participate, teaching styles and experiences of the intervention. Interviews explored views on implementation and dissemination. Framework analysis was used to analyse data. Results: Qualitative data elicited three themes associated with the delivery of BGDP that affected implementation: project design, session content and project organisation. 'Project design' found issues associated with recruitment, timetabling and session quantity to influence the effectiveness of BGDP. 'Session content' found that dance instructors delivered a range of content and that girls enjoyed a variety of dance. Themes within 'project organisation' suggested an 'open enrolment' policy and greater parental involvement may facilitate better attendance. Conclusions: After-school PA interventions have potential for increasing PA levels among adolescent girls. There is a need to consider the context in which interventions are delivered and implement them in ways that are appropriate to the needs of participants

The politicisation of evaluation: constructing and contesting EU policy performance

Although systematic policy evaluation has been conducted for decades and has been growing strongly within the European Union (EU) institutions and in the member states, it remains largely underexplored in political science literatures. Extant work in political science and public policy typically focuses on elements such as agenda setting, policy shaping, decision making, or implementation rather than evaluation. Although individual pieces of research on evaluation in the EU have started to emerge, most often regarding policy “effectiveness” (one criterion among many in evaluation), a more structured approach is currently missing. This special issue aims to address this gap in political science by focusing on four key focal points: evaluation institutions (including rules and cultures), evaluation actors and interests (including competencies, power, roles and tasks), evaluation design (including research methods and theories, and their impact on policy design and legislation), and finally, evaluation purpose and use (including the relationships between discourse and scientific evidence, political attitudes and strategic use). The special issue considers how each of these elements contributes to an evolving governance system in the EU, where evaluation is playing an increasingly important role in decision making

Realist evaluation of an enhanced health visiting programme

The health visitors' role in many countries is changing. In Scotland, the role has undergone substantial changes through the introduction of an enhanced health visiting programme, which includes increased, structured home visits. This evaluation was conducted within NHS Ayrshire and Arran, one of the 14 Scottish Health Boards. Our aim was to understand and explain how, and why, the programme could contribute to improving health and wellbeing outcomes for children and families.We used a realist evaluation approach, conducted in three phases. In phase one, eight managerial staff involved in developing and implementing the programme provided data, which were used to develop initial programme theories. In phase two, the programme theories were tested using qualitative data from 25 health visitors and 22 parents. The programme theories were refined through analyses and interpretation of data in phase three.The home visiting context provided by the programme interacted with the mechanisms of the programme and produced outcomes such as early identification of health and wellbeing issues amongst families who needed more support, leading to referral and engagement with sources of additional help. The home visits facilitated development of parent-health visitor relationships, and parents considered health visitors as their first point of contact on children's wellbeing and developmental-related issues. Moreover, the programme provided more clarity to health visitors' role, which in turn enhanced partnership working. However, there were aspects of the programme that may require further development. For instance, both parents and health visitors were concerned about the wide gaps between some home visits.The enhanced health visiting programme increased opportunities for monitoring and early identification of health and wellbeing concerns. It created structures for a more efficient partnership working and ensured that the needs of children and families were supported. These benefits need to be evaluated further in an effectiveness study

Who needs what from a national health research system: Lessons from reforms to the English Department of Health's R&D system

This article has been made available through the Brunel Open Access Publishing Fund.Health research systems consist of diverse groups who have some role in health research, but the boundaries around such a system are not clear-cut. To explore what various stakeholders need we reviewed the literature including that on the history of English health R&D reforms, and we also applied some relevant conceptual frameworks. We first describe the needs and capabilities of the main groups of stakeholders in health research systems, and explain key features of policymaking systems within which these stakeholders operate in the UK. The five groups are policymakers (and health care managers), health professionals, patients and the general public, industry, and researchers. As individuals and as organisations they have a range of needs from the health research system, but should also develop specific capabilities in order to contribute effectively to the system and benefit from it. Second, we discuss key phases of reform in the development of the English health research system over four decades - especially that of the English Department of Health's R&D system - and identify how far legitimate demands of key stakeholder interests were addressed. Third, in drawing lessons we highlight points emerging from contemporary reports, but also attempt to identify issues through application of relevant conceptual frameworks. The main lessons are: the importance of comprehensively addressing the diverse needs of various interacting institutions and stakeholders; the desirability of developing facilitating mechanisms at interfaces between the health research system and its various stakeholders; and the importance of additional money in being able to expand the scope of the health research system whilst maintaining support for basic science. We conclude that the latest health R&D strategy in England builds on recent progress and tackles acknowledged weaknesses. The strategy goes a considerable way to identifying and more effectively meeting the needs of key groups such as medical academics, patients and industry, and has been remarkably successful in increasing the funding for health research. There are still areas that might benefit from further recognition and resourcing, but the lessons identified, and progress made by the reforms are relevant for the design and coordination of national health research systems beyond England.This article is available through the Brunel Open Access Publishing Fund

Factors Influencing Engagement, Perceived Usefulness and Behavioral Mechanisms Associated with a Text Message Support Program

Introduction Many studies have now demonstrated the efficacy of text messaging in positively changing behaviours. We aimed to identify features and factors that explain the effectiveness of a successful text messaging program in terms of user engagement, perceived usefulness, behavior change and program delivery preferences. Methods Mixed methods qualitative design combining four data sources; (i) analytic data extracted directly from the software system, (ii) participant survey, (iii) focus groups to identify barriers and enablers to implementation and mechanisms of effect and (iv) recruitment screening logs and text message responses to examine engagement. This evaluation was conducted within the TEXT ME trial—a parallel design, single-blind randomized controlled trial (RCT) of 710 patients with coronary heart disease (CHD). Qualitative data were interpreted using inductive thematic analysis. Results 307/352 (87% response rate) of recruited patients with CHD completed the program evaluation survey at six months and 25 participated in a focus group. Factors increasing engagement included (i) ability to save and share messages, (ii) having the support of providers and family, (iii) a feeling of support through participation in the program, (iv) the program being initiated close to the time of a cardiovascular event, (v) personalization of the messages, (vi) opportunity for initial face-to-face contact with a provider and (vii) that program and content was perceived to be from a credible source. Clear themes relating to program delivery were that diet and physical activity messages were most valued, four messages per week was ideal and most participants felt program duration should be provided for at least for six months or longer. Conclusions This study provides context and insight into the factors influencing consumer engagement with a text message program aimed at improving health-related behavior. The study suggests program components that may enhance potential success but will require integration at the development stage to optimize up-scaling

Mapping differential interactomes by affinity purification coupled with data independent mass spectrometry acquisition

Characterizing changes in protein-protein interactions associated with sequence variants (e.g. disease-associated mutations or splice forms) or following exposure to drugs, growth factors or hormones is critical to understanding how protein complexes are built, localized and regulated. Affinity purification (AP) coupled with mass spectrometry permits the analysis of protein interactions under near-physiological conditions, yet monitoring interaction changes requires the development of a robust and sensitive quantitative approach, especially for large-scale studies where cost and time are major considerations. To this end, we have coupled AP to data-independent mass spectrometric acquisition (SWATH), and implemented an automated data extraction and statistical analysis pipeline to score modulated interactions. Here, we use AP-SWATH to characterize changes in protein-protein interactions imparted by the HSP90 inhibitor NVP-AUY922 or melanoma-associated mutations in the human kinase CDK4. We show that AP-SWATH is a robust label-free approach to characterize such changes, and propose a scalable pipeline for systems biology studies

Rosetta FlexPepDock ab-initio: Simultaneous Folding, Docking and Refinement of Peptides onto Their Receptors

Flexible peptides that fold upon binding to another protein molecule mediate a large number of regulatory interactions in the living cell and may provide highly specific recognition modules. We present Rosetta FlexPepDock ab-initio, a protocol for simultaneous docking and de-novo folding of peptides, starting from an approximate specification of the peptide binding site. Using the Rosetta fragments library and a coarse-grained structural representation of the peptide and the receptor, FlexPepDock ab-initio samples efficiently and simultaneously the space of possible peptide backbone conformations and rigid-body orientations over the receptor surface of a given binding site. The subsequent all-atom refinement of the coarse-grained models includes full side-chain modeling of both the receptor and the peptide, resulting in high-resolution models in which key side-chain interactions are recapitulated. The protocol was applied to a benchmark in which peptides were modeled over receptors in either their bound backbone conformations or in their free, unbound form. Near-native peptide conformations were identified in 18/26 of the bound cases and 7/14 of the unbound cases. The protocol performs well on peptides from various classes of secondary structures, including coiled peptides with unusual turns and kinks. The results presented here significantly extend the scope of state-of-the-art methods for high-resolution peptide modeling, which can now be applied to a wide variety of peptide-protein interactions where no prior information about the peptide backbone conformation is available, enabling detailed structure-based studies and manipulation of those interactions