Systematic evaluation of implementation fidelity of complex interventions in health and social care

<p>Abstract</p> <p>Background</p> <p>Evaluation of an implementation process and its fidelity can give insight into the 'black box' of interventions. However, a lack of standardized methods for studying fidelity and implementation process have been reported, which might be one reason for the fact that few prior studies in the field of health service research have systematically evaluated interventions' implementation processes.</p> <p>The aim of this project is to systematically evaluate implementation fidelity and possible factors influencing fidelity of complex interventions in health and social care.</p> <p>Methods</p> <p>A modified version of The Conceptual Framework for Implementation Fidelity will be used as a conceptual model for the evaluation. The modification implies two additional moderating factors: context and recruitment. A systematic evaluation process was developed. Multiple case study method is used to investigate implementation of three complex health service interventions. Each case will be investigated in depth and longitudinally, using both quantitative and qualitative methods.</p> <p>Discussion</p> <p>This study is the first attempt to empirically test The Conceptual Framework for Implementation Fidelity. The study can highlight mechanism and factors of importance when implementing complex interventions. Especially the role of the moderating factors on implementation fidelity can be clarified.</p> <p>Trial Registration</p> <p>Supported Employment, SE, among people with severe mental illness -- a randomized controlled trial: NCT00960024.</p

Assessing the organizational context for EBP implementation: the development and validity testing of the Implementation Climate Scale (ICS)

BACKGROUND: Although the importance of the organizational environment for implementing evidence-based practices (EBP) has been widely recognized, there are limited options for measuring implementation climate in public sector health settings. The goal of this research was to develop and test a measure of EBP implementation climate that would both capture a broad range of issues important for effective EBP implementation and be of practical use to researchers and managers seeking to understand and improve the implementation of EBPs. METHODS: Participants were 630 clinicians working in 128 work groups in 32 US-based mental health agencies. Items to measure climate for EBP implementation were developed based on past literature on implementation climate and other strategic climates and in consultation with experts on the implementation of EBPs in mental health settings. The sample was randomly split at the work group level of analysis; half of the sample was used for exploratory factor analysis (EFA), and the other half was used for confirmatory factor analysis (CFA). The entire sample was utilized for additional analyses assessing the reliability, support for level of aggregation, and construct-based evidence of validity. RESULTS: The EFA resulted in a final factor structure of six dimensions for the Implementation Climate Scale (ICS): 1) focus on EBP, 2) educational support for EBP, 3) recognition for EBP, 4) rewards for EBP, 5) selection for EBP, and 6) selection for openness. This structure was supported in the other half of the sample using CFA. Additional analyses supported the reliability and construct-based evidence of validity for the ICS, as well as the aggregation of the measure to the work group level. CONCLUSIONS: The ICS is a very brief (18 item) and pragmatic measure of a strategic climate for EBP implementation. It captures six dimensions of the organizational context that indicate to employees the extent to which their organization prioritizes and values the successful implementation of EBPs. The ICS can be used by researchers to better understand the role of the organizational context on implementation outcomes and by organizations to evaluate their current climate as they consider how to improve the likelihood of implementation success. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s13012-014-0157-1) contains supplementary material, which is available to authorized users

A conceptual framework for implementation fidelity

<p>Abstract</p> <p>Background</p> <p>Implementation fidelity refers to the degree to which an intervention or programme is delivered as intended. Only by understanding and measuring whether an intervention has been implemented with fidelity can researchers and practitioners gain a better understanding of how and why an intervention works, and the extent to which outcomes can be improved.</p> <p>Discussion</p> <p>The authors undertook a critical review of existing conceptualisations of implementation fidelity and developed a new conceptual framework for understanding and measuring the process. The resulting theoretical framework requires testing by empirical research.</p> <p>Summary</p> <p>Implementation fidelity is an important source of variation affecting the credibility and utility of research. The conceptual framework presented here offers a means for measuring this variable and understanding its place in the process of intervention implementation.</p

Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care

<p>Abstract</p> <p>Background</p> <p>Prior studies measuring fidelity of complex interventions have mainly evaluated adherence, and not taken factors affecting adherence into consideration. A need for studies that clarify the concept of fidelity and the function of factors moderating fidelity has been emphasized. The aim of the study was to systematically evaluate implementation fidelity and possible factors influencing fidelity of a complex care continuum intervention for frail elderly people.</p> <p>Methods</p> <p>The intervention was a systematization of the collaboration between a nurse with geriatric expertise situated at the emergency department, the hospital ward staff, and a multi-professional team with a case manager in the municipal care services for older people. Implementation was evaluated between September 2008 and May 2010 with observations of work practices, stakeholder interviews, and document analysis according to a modified version of The Conceptual Framework for Implementation Fidelity.</p> <p>Results</p> <p>A total of 16 of the 18 intervention components were to a great extent delivered as planned, while some new components were added to the model. No changes in the frequency or duration of the 18 components were observed, but the dose of the added components varied over time. Changes in fidelity were caused in a complex, interrelated fashion by all the moderating factors in the framework, i.e., context, staff and participant responsiveness, facilitation, recruitment, and complexity.</p> <p>Discussion</p> <p>The Conceptual Framework for Implementation Fidelity was empirically useful and included comprehensive measures of factors affecting fidelity. Future studies should focus on developing the framework with regard to how to investigate relationships between the moderating factors and fidelity over time.</p> <p>Trial registration</p> <p>ClinicalTrials.gov, <a href="http://www.clinicaltrials.gov/ct2/show/NCT01260493">NCT01260493</a>.</p

Study protocol for a group randomized controlled trial of a classroom-based intervention aimed at preventing early risk factors for drug abuse: integrating effectiveness and implementation research

<p>Abstract</p> <p>Background</p> <p>While a number of preventive interventions delivered within schools have shown both short-term and long-term impact in epidemiologically based randomized field trials, programs are not often sustained with high-quality implementation over time. This study was designed to support two purposes. The first purpose was to test the effectiveness of a universal classroom-based intervention, the Whole Day First Grade Program (WD), aimed at two early antecedents to drug abuse and other problem behaviors, namely, aggressive, disruptive behavior and poor academic achievement. The second purpose--the focus of this paper--was to examine the utility of a multilevel structure to support high levels of implementation during the effectiveness trial, to sustain WD practices across additional years, and to train additional teachers in WD practices.</p> <p>Methods</p> <p>The WD intervention integrated three components, each previously tested separately: classroom behavior management; instruction, specifically reading; and family-classroom partnerships around behavior and learning. Teachers and students in 12 schools were randomly assigned to receive either the WD intervention or the standard first-grade program of the school system (SC). Three consecutive cohorts of first graders were randomized within schools to WD or SC classrooms and followed through the end of third grade to test the effectiveness of the WD intervention. Teacher practices were assessed over three years to examine the utility of the multilevel structure to support sustainability and scaling-up.</p> <p>Discussion</p> <p>The design employed in this trial appears to have considerable utility to provide data on WD effectiveness and to inform the field with regard to structures required to move evidence-based programs into practice.</p> <p>Trial Registration</p> <p><b>Clinical Trials Registration Number</b>: NCT00257088</p

A typology of practice narratives during the implementation of a preventive, community intervention trial

<p>Abstract</p> <p>Background</p> <p>Traditional methods of process evaluation encompass what components were delivered, but rarely uncover how practitioners position themselves and act relative to an intervention being tested. This could be crucial for expanding our understanding of implementation and its contribution to intervention effectiveness.</p> <p>Methods</p> <p>We undertook a narrative analysis of in-depth, unstructured field diaries kept by nine community development practitioners for two years. The practitioners were responsible for implementing a multi-component, preventive, community-level intervention for mothers of new babies in eight communities, as part of a cluster randomised community intervention trial. We constructed a narrative typology of approaches to practice, drawing on the phenomenology of Alfred Schutz and Max Weber's Ideal Type theory.</p> <p>Results</p> <p>Five types of practice emerged, from a highly 'technology-based' type that was faithful to intervention specifications, through to a 'romantic' type that held relationships to be central to daily operations, with intact relationships being the final arbiter of intervention success. The five types also differed in terms of how others involved in the intervention were characterized, the narrative form (<it>e.g</it>., tragedy, satire) and where and how transformative change in communities was best created. This meant that different types traded-off or managed the priorities of the intervention differently, according to the deeply held values of their type.</p> <p>Conclusions</p> <p>The data set constructed for this analysis is unique. It revealed that practitioners not only exercise their agency within interventions, they do so systematically, that is, according to a pattern. The typology is the first of its kind and, if verified through replication, may have value for anticipating intervention dynamics and explaining implementation variation in community interventions.</p

Information and digital literacies; a review of concepts

A detailed literature reviewing, analysing the multiple and confusing concepts around the ideas of information literacy and digital literacy at the start of the millennium. The article was well-received, and is my most highly-cited work, with over 1100 citations