The Mechanisms of Implementation Support - Findings from a Systematic Integrative Review

Purpose: Mechanisms of Change (MoC) explain how strategies used to enhance the uptake of evidence in social and human services enable change in the behaviors of individual practitioners, organizational leaders or entire organizations, and systems. One such strategy is the use of implementation support practitioner (ISPs). This study examines the mechanisms through which ISPs facilitate behavior supportive of the implementation of research-supported interventions. Methods: A systematic, integrative review was conducted. The conceptualization of MoCs built on a model by Dalkin and colleagues. Results: Based on a unique combination of knowledge, skills, and attitudes, ISPs install trust in and among their stakeholders and utilize this trust to promote meaningful and relevant learning; provide ongoing opportunities for learning, reflection, and support; help to span boundaries; and positively motivate stakeholders. Discussion: ISPs do not represent a short cut to better implementation. They represent an additional implementation challenge that requires dedicated attention and resources

Knowledge and attitudes of Implementation Support Practitioners—Findings from a systematic integrative review

Background: It requires thoughtful planning and work to successfully apply and sustain research-supported interventions like healthcare treatments, social support, or preventive programs in practice. Implementation support practitioners (ISPs) such as facilitators, technical assistance providers, knowledge brokers, coaches or consultants may be involved to actively support the implementation process. This article presents knowledge and attitudes ISPs bring to their work. Methods: Building on a previously developed program logic, a systematic integrative review was conducted. Literature was sourced by searching nine electronic data bases, organizational websites, and by launching a call for publications among selected experts and social media. Article screening was performed independently by two researchers, and data from included studies were extracted by members of the research team and quality-assured by the lead researcher. The quality of included RCTs was assessed based on a framework by Hodder and colleagues. Thematic Analysis was used to capture information on knowledge and attitudes of ISPs across the included studies. Euler diagrams and heatmaps were used to present the results. Results: Results are based on 79 included studies. ISPs reportedly displayed knowledge about the clinical practice they work with, implementation / improvement practice, the local context, supporting change processes, and facilitating evidence-based practice in general. In particular, knowledge about the intervention to be implemented and its target population, specific improvement / implementation methods and approaches, organizational structures and sensitivities, training, and characteristics of (good) research was described in the literature. Seven themes describing ISPs’ attitudes were identified: 1) professional, 2) motivated / motivating / encouraging / empowering, 3) empathetic / respectful / sensitive, 4) collaborative / inclusive, 5) authentic, 6) creative / flexible / innovative / adaptive, and 7) frank / direct / honest. Pertaining to a professional attitude, being responsive and focused were the most prevalent indicators across included publications. Conclusion: The wide range and complexity of knowledge and attitudes found in the literature calls for a comprehensive and systematic approach to collaboratively develop a professional role for ISPs across disciplines. Embedding the ISP role in different health and social welfare settings will enhance implementation capacities considerably

The Mechanisms of Implementation Support - Findings from a Systematic Integrative Review

Purpose: Mechanisms of Change (MoC) explain how strategies used to enhance the uptake of evidence in social and human services enable change in the behaviors of individual practitioners, organizational leaders or entire organizations, and systems. One such strategy is the use of implementation support practitioner (ISPs). This study examines the mechanisms through which ISPs facilitate behavior supportive of the implementation of research-supported interventions. Methods: A systematic, integrative review was conducted. The conceptualization of MoCs built on a model by Dalkin and colleagues. Results: Based on a unique combination of knowledge, skills, and attitudes, ISPs install trust in and among their stakeholders and utilize this trust to promote meaningful and relevant learning; provide ongoing opportunities for learning, reflection, and support; help to span boundaries; and positively motivate stakeholders. Discussion: ISPs do not represent a short cut to better implementation. They represent an additional implementation challenge that requires dedicated attention and resources

The feasibility of using Apple's ResearchKit for recruitment and data collection: Considerations for mental health research

In 2015, Apple launched an open-source software framework called ResearchKit. ResearchKit provides an infrastructure for conducting remote, smartphone-based research trials through the means of Apple's App Store. Such trials may have several advantages over conventional trial methods including the removal of geographic barriers, frequent assessments of participants in real-life settings, and increased inclusion of seldom-heard communities. The aim of the current study was to explore the feasibility of participant recruitment and the potential for data collection in the non-clinical population in a smartphone-based trial using ResearchKit. As a case example, an app called eMovit, a behavioural activation (BA) app with the aim of helping users to build healthy habits was used. The study was conducted over a 9-month period. Any iPhone user with access to the App Stores of The Netherlands, Belgium, and Germany could download the app and participate in the study. During the study period, the eMovit app was disseminated amongst potential users via social media posts (Twitter, Facebook, LinkedIn), paid social media advertisements (Facebook), digital newsletters and newspaper articles, blogposts and other websites. In total, 1,788 individuals visited the eMovit landing page. A total of 144 visitors subsequently entered Apple's App Store through that landing page. The eMovit product page was viewed 10,327 times on the App Store. With 79 installs, eMovit showed a conversion rate of 0.76% from product view to install of the app. Of those 79 installs, 53 users indicated that they were interested to participate in the research study and 36 subsequently consented and completed the demographics and the participants quiz. Fifteen participants completed the first PHQ-8 assessment and one participant completed the second PHQ-8 assessment. We conclude that from a technological point of view, the means provided by ResearchKit are well suited to be integrated into the app process and thus facilitate conducting smartphone-based studies. However, this study shows that although participant recruitment is technically straightforward, only low recruitment rates were achieved with the dissemination strategies applied. We argue that smartphone-based trials (using ResearchKit) require a well-designed app dissemination process to attain a sufficient sample size. Guidelines for smartphone-based trial designs and recommendations on how to work with challenges of mHealth research will ensure the quality of these trials, facilitate researchers to do more testing of mental health apps and with that enlarge the evidence-base for mHealth

Towards an Implementation-STakeholder Engagement Model (I-STEM) for improving health and social care services

BACKGROUND: The implementation science literature acknowledges a need for engagement of key stakeholders when designing, delivering and evaluating implementation work. To date, the literature reports minimal or focused stakeholder engagement, where stakeholders are engaged in either barrier identification and/or barrier prioritisation. This paper begins to answer calls from the literature for the development of tools and guidance to support comprehensive stakeholder engagement in implementation research and practice. The paper describes the systematic development of the Implementation-STakeholder Engagement Model (I-STEM) in the context of an international, large-scale empirical implementation study (ImpleMentAll) aimed at evaluating the effectiveness of a tailored implementation toolkit. The I-STEM is a sensitising tool that defines key considerations and activities for undertaking stakeholder engagement activities across an implementation process. METHODS: In-depth, semistructured interviews and observations were conducted with implementers who were tailoring implementation strategies to integrate and embed internet-based cognitive behavioural therapy (iCBT) services in 12 routine mental health care organisations in nine countries in Europe and Australia. The analytical process was informed by principles of first- and third-generation Grounded Theory, including constant comparative method. RESULTS: We conducted 55 interviews and observed 19 implementation-related activities (e.g., team meetings and technical support calls). The final outcome of our analysis is expressed in an initial version of the I-STEM, consisting of five interrelated concepts: engagement objectives, stakeholder mapping, engagement approaches, engagement qualities and engagement outcomes. Engagement objectives are goals that implementers plan to achieve by working with stakeholders in the implementation process. Stakeholder mapping involves identifying a range of organisations, groups or people who may be instrumental in achieving the engagement objectives. Engagement approaches define the type of work that is undertaken with stakeholders to achieve the engagement objectives. Engagement qualities define the logistics of the engagement approach. Lastly, every engagement activity may result in a range of engagement outcomes. CONCLUSION: The I-STEM represents potential avenues for substantial stakeholder engagement activity across key phases of an implementation process. It provides a conceptual model for the planning, delivery, evaluation and reporting of stakeholder engagement activities. The I-STEM is nonprescriptive and highlights the importance of a flexible, iterative approach to stakeholder engagement. It is developmental and will require application and validation across a range of implementation activities. PATIENT OR PUBLIC CONTRIBUTION: Patient contribution to ImpleMentAll trial was facilitated by GAMIAN-Europe at all stages-from grant development to dissemination. GAMIAN-Europe brings together a wide variety of patient representation organisations (local, regional and national) from almost all European countries. GAMIAN-Europe was involved in pilot testing the ItFits-toolkit and provided their views on the various aspects, including stakeholder engagement. Patients were also represented in the external advisory board providing support and advice on the design, conduct and interpretation of the wider project, including the development of the ItFits-toolkit. TRIAL REGISTRATION: ClinicalTrials.gov NCT03652883. Retrospectively registered on 29 August 2018

Effectiveness of Self-guided Tailored Implementation Strategies in Integrating and Embedding Internet-Based Cognitive Behavioral Therapy in Routine Mental Health Care: Results of a Multicenter Stepped-Wedge Cluster Randomized Trial

BACKGROUND: Internet-based cognitive behavioral therapy (iCBT) services for common mental health disorders have been found to be effective. There is a need for strategies that improve implementation in routine practice. One-size-fits-all strategies are likely to be ineffective. Tailored implementation is considered as a promising approach. The self-guided integrated theory-based Framework for intervention tailoring strategies toolkit (ItFits-toolkit) supports local implementers in developing tailored implementation strategies. Tailoring involves identifying local barriers; matching selected barriers to implementation strategies; developing an actionable work plan; and applying, monitoring, and adapting where necessary. OBJECTIVE: This study aimed to compare the effectiveness of the ItFits-toolkit with implementation-as-usual (IAU) in implementing iCBT services in 12 routine mental health care organizations in 9 countries in Europe and Australia. METHODS: A stepped-wedge cluster randomized trial design with repeated measures was applied. The trial period lasted 30 months. The primary outcome was the normalization of iCBT delivery by service providers (therapists, referrers, IT developers, and administrators), which was measured with the Normalization Measure Development as a proxy for implementation success. A 3-level linear mixed-effects modeling was applied to estimate the effects. iCBT service uptake (referral and treatment completion rates) and implementation effort (hours) were used as secondary outcomes. The perceived satisfaction (Client Satisfaction Questionnaire), usability (System Usability Scale), and impact of the ItFits-toolkit by implementers were used to assess the acceptability of the ItFits-toolkit. RESULTS: In total, 456 mental health service providers were included in this study. Compared with IAU, the ItFits-toolkit had a small positive statistically significant effect on normalization levels in service providers (mean 0.09, SD 0.04; P=.02; Cohen d=0.12). The uptake of iCBT by patients was similar to that of IAU. Implementers did not spend more time on implementation work when using the ItFits-toolkit and generally regarded the ItFits-toolkit as usable and were satisfied with it. CONCLUSIONS: The ItFits-toolkit performed better than the usual implementation activities in implementing iCBT services in routine practice. There is practical utility in the ItFits-toolkit for supporting implementers in developing and applying effective tailored implementation strategies. However, the effect on normalization levels among mental health service providers was small. These findings warrant modesty regarding the effectiveness of self-guided tailored implementation of iCBT services in routine practice. TRIAL REGISTRATION: ClinicalTrials.gov NCT03652883; https://clinicaltrials.gov/ct2/show/NCT03652883. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.1186/s13063-020-04686-4

Tailored implementation of internet-based cognitive behavioural therapy in the multinational context of the ImpleMentAll project: a study protocol for a stepped wedge cluster randomized trial.

BACKGROUND: Internet-based Cognitive Behavioural Therapy (iCBT) is found effective in treating common mental disorders. However, the use of these interventions in routine care is limited. The international ImpleMentAll study is funded by the European Union's Horizon 2020 programme. It is concerned with studying and improving methods for implementing evidence-based iCBT services for common mental disorders in routine mental health care. A digitally accessible implementation toolkit (ItFits-toolkit) will be introduced to mental health care organizations with the aim to facilitate the ongoing implementation of iCBT services within local contexts. This study investigates the effectiveness of the ItFits-toolkit by comparing it to implementation-as-usual activities. METHODS: A stepped wedge cluster randomized controlled trial (SWT) design will be applied. Over a trial period of 30 months, the ItFits-toolkit will be introduced sequentially in twelve routine mental health care organizations in primary and specialist care across nine countries in Europe and Australia. Repeated measures are applied to assess change over time in the outcome variables. The effectiveness of the ItFits-toolkit will be assessed in terms of the degree of normalization of the use of the iCBT services. Several exploratory outcomes including uptake of the iCBT services will be measured to feed the interpretation of the primary outcome. Data will be collected via a centralized data collection system and analysed using generalized linear mixed modelling. A qualitative process evaluation of routine implementation activities and the use of the ItFits-toolkit will be conducted within this study. DISCUSSION: The ImpleMentAll study is a large-scale international research project designed to study the effectiveness of tailored implementation. Using a SWT design that allows to examine change over time, this study will investigate the effect of tailored implementation on the normalization of the use of iCBT services and their uptake. It will provide a better understanding of the process and methods of tailoring implementation strategies. If found effective, the ItFits-toolkit will be made accessible for mental health care service providers, to help them overcome their context-specific implementation challenges. TRIAL REGISTRATION: ClinicalTrials.gov NCT03652883 . Retrospectively registered on 29 August 2018

Electronic Results Addendum (ERA 2): The Mechanisms and Determinants of Implementation Support – Findings from a Systematic Integrative Review

This Electronic Results Addendum (ERA 2) provides a detailed overview of data extraction and analysis across included studies

Health technology assessment frameworks for eHealth:A systematic review

Objectives Traditionally, health technology assessment (HTA) focuses on assessing the impact of pharmaceutical technologies on health and care. Resources are scarce and policy makers aim to achieve effective, accessible health care. eHealth innovations are increasingly more integrated in all healthcare domains. However, how eHealth is assessed prior to its implementation in care practices is unclear. To support evidence-informed policy making, this study aimed to identify frameworks and methods for assessing eHealth's impact on health care.Methods The scientific literature in five bibliographical databases was systematically reviewed. Articles were included if the study was conducted in a clinical setting, used an HTA framework and assessed an eHealth service. A systematic qualitative narrative approach was applied for analysis and reporting.Results Twenty-one HTA frameworks were identified in twenty-three articles. All frameworks addressed outcomes related to the technical performance and functionalities of eHealth service under assessment. The majority also addressed costs (n = 19), clinical outcomes (n = 14), organizational (n = 15) and system level aspects (n = 13). Most frameworks can be classified as dimensional (n = 13), followed by staged (n = 3), hybrid (n = 3), and business modeling frameworks (n = 2). Six frameworks specified assessment outcomes and methods.Conclusions HTA frameworks are available for a-priori impact assessment of eHealth services. The frameworks vary in assessment outcomes, methods, and specificity. Demonstrated applicability in practice is limited. Recommendations include standardization of: (i) reporting characteristics of eHealth services, and (ii) specifying assessment outcomes and methods following a stepped-approach tailored to the functional characteristics of eHealth services. Standardization might improve the quality and comparability of eHTA assessments

Implementation Support Skills: Findings From a Systematic Integrative Review

Purpose: Skills in selecting and designing strategies for implementing research-supported interventions (RSIs) within specific local contexts are important for progressing a wider RSI adoption and application in human and social services. This also applies to a particular role in implementation, the implementation support practitioner (ISP). This study examines which strategies have been reported as being used by ISPs across multiple bodies of research on implementation support and how these strategies were applied in concrete practice settings. Methods: A systematic integrative review was conducted. Data analysis utilized the Expert Recommendations for Implementing Change compilation of implementation strategies. Results: Studies reported on 18 implementation strategies commonly used by different ISPs, who require mastery in selecting, operationalizing, and detailing these. Two further strategies not included in the ERIC compilation could be identified. Discussion: Given the use of primarily more feasible implementation support strategies among ISPs, their potential as agents of change may be underutilized