Cost and feasibility: an exploratory case study comparing use of a literature review method with questionnaires, interviews and focus groups to identify barriers for a behaviour-change intervention

Background: It is often recommended that behaviour-change interventions be tailored to barriers. There is a scarcity of research into the best method of barrier identification, although combining methods has been suggested to be beneficial. This paper compares the feasibility and costs of three different methods of barrier identification used in three implementation projects conducted in primary care. Methods: Underpinned by a theory-base, project one used a questionnaire and interviews; project two used a single focus group and questionnaire, and project three used a literature review of published barriers. The feasibility of each project, as experienced by the research team, and labour costs are summarised. Results: The literature review of published barriers was the least costly and most feasible method, being quick to conduct and avoiding the challenges of recruitment experienced when using interviews or a questionnaire. The feasibility of using questionnaires was further reduced by the time taken develop the instruments. Conducting a single focus group was also found to be a more feasible method, taking less time than interviews to collect and analyse the barriers. Conclusions: Considering the ease of recruitment, time required and cost of the different methods to collect barriers is crucial at the start of implementation studies. The literature review method is the least costly and most feasible method. Use of a single focus group was found to be more feasible than conducting individual interviews or administering a questionnaire, with less recruitment challenges experienced, and quicker data collection. Future research would benefit from comparing the robustness of the methods in terms of the comprehensiveness of barriers identified

Challenges in identifying barriers to adoption in a theory-based implementation study: lessons for future implementation studies

Background: Exploring barriers to the uptake of research based recommendations into practice is an important part of the development of implementation programmes. Techniques to identify barriers can include use of theory-informed questionnaires and qualitative interviews. Conceptualising and measuring theory-informed factors, and engaging health professionals’ to uncover all potential barriers, can be a difficult task. This paper presents a case study of the process of trying to identify, systematically, the key factors influencing health professionals’ referrals for women diagnosed with mild to moderate postnatal depression for psychological treatment. The paper illustrates how the factors were conceptualised and measured and explores the real world challenges experienced, with implications for future implementation studies. Methods: Theory-informed factors were conceptualised and measured using a questionnaire and interviews. The questionnaire was piloted, before being administered to general practitioners, practice nurses and health visitors working in general practices in one area of the UK NHS. The interviews were conducted with a small sample of general practitioners who had not completed the questionnaire, further exploring factors influencing their referral decisions in the local context. Results: The response rate to the questionnaire was low (19%), despite selecting the recommendation to target through engagement with local stakeholders and surveying local health professionals, and despite using two reminders, an incentive prize, and phone calls to practice managers to bolster response rates. Conclusions: Two significant challenges to achieving higher response rates and successfully exploring local context were identified: the difficulties of developing a robust- but feasible- questionnaire to explore theory-informed factors, and targeting recommendations that are important to policy makers, but which health professionals view as unimportant. This case study highlights the “trade-off” between scientifically rigorous collection of data against the pragmatism and flexibility requirements of “real world” implementation. Future implementation studies should explore different ways of identifying factors influencing the adoption of recommendations to bridge this gulf

Immediate versus sustained effects: interrupted time series analysis of a tailored intervention

Background: Detailed intervention descriptions and robust evaluations that test intervention impact—and explore reasons for impact—are an essential part of progressing implementation science. Time series designs enable the impact and sustainability of intervention effects to be tested. When combined with time series designs, qualitative methods can provide insight into intervention effectiveness and help identify areas for improvement for future interventions. This paper describes the development, delivery, and evaluation of a tailored intervention designed to increase primary health care professionals’ adoption of a national recommendation that women with mild to moderate postnatal depression (PND) are referred for psychological therapy as a first stage treatment. Methods: Three factors influencing referral for psychological treatment were targeted using three related intervention components: a tailored educational meeting, a tailored educational leaflet, and changes to an electronic system data template used by health professionals during consultations for PND. Evaluation comprised time series analysis of monthly audit data on percentage referral rates and monthly first prescription rates for anti-depressants. Interviews were conducted with a sample of health professionals to explore their perceptions of the intervention components and to identify possible factors influencing intervention effectiveness. Results: The intervention was associated with a significant, immediate, positive effect upon percentage referral rates for psychological treatments. This effect was not sustained over the ten month follow-on period. Monthly rates of anti-depressant prescriptions remained consistently high after the intervention. Qualitative interview findings suggest key messages received from the intervention concerned what appropriate antidepressant prescribing is, suggesting this to underlie the lack of impact upon prescribing rates. However, an understanding that psychological treatment can have long-term benefits was also cited. Barriers to referral identified before intervention were cited again after the intervention, suggesting the intervention had not successfully tackled the barriers targeted. Conclusion: A time series design allowed the initial and sustained impact of our intervention to be tested. Combined with qualitative interviews, this provided insight into intervention effectiveness. Future research should test factors influencing intervention sustainability, and promote adoption of the targeted behavior and dis-adoption of competing behaviors where appropriate

Translating research into practice in Leeds and Bradford (TRiPLaB): a protocol for a programme of research

Background The National Institute for Health Research (NIHR) has funded nine Collaborations for Leadership in Applied Health Research and Care (CLAHRCs). Each CLAHRC is a partnership between higher education institutions (HEIs) and the NHS in nine UK regional health economies. The CLAHRC for Leeds, York, and Bradford comprises two 'research themes' and three 'implementation themes.' One of these implementation themes is Translating Research into Practice in Leeds and Bradford (TRiPLaB). TRiPLaB aims to develop, implement, and evaluate methods for inducing and sustaining the uptake of research knowledge into practice in order to improve the quality of health services for the people of Leeds and Bradford. Methods TRiPLaB is built around a three-stage, sequential, approach using separate, longitudinal case studies conducted with collaborating NHS organisations, TRiPLaB will select robust innovations to implement, conduct a theory-informed exploration of the local context using a variety of data collection and analytic methods, and synthesise the information collected to identify the key factors influencing the uptake and adoption of targeted innovations. This synthesis will inform the development of tailored, multifaceted, interventions designed to increase the translation of research findings into practice. Mixed research methods, including time series analysis, quasi-experimental comparison, and qualitative process evaluation, will be used to evaluate the impact of the implementation strategies deployed. Conclusion TRiPLaB is a theory-informed, systematic, mixed methods approach to developing and evaluating tailored implementation strategies aimed at increasing the translation of research-based findings into practice in one UK health economy. Through active collaboration with its local NHS, TRiPLaB aims to improve the quality of health services for the people of Leeds and Bradford and to contribute to research knowledge regarding the interaction between context and adoption behaviour in health services

Gathering opinion leader data for a tailored implementation intervention in secondary healthcare: a randomised trial

Background: Health professionals’ behaviour is a key component in compliance with evidence-based recommendations. Opinion leaders are an oft-used method of influencing such behaviours in implementation studies, but reliably and cost effectively identifying them is not straightforward. Survey and questionnaire based data collection methods have potential and carefully chosen items can – in theory – both aid identification of opinion leaders and help in the design of an implementation strategy itself. This study compares two methods of identifying opinion leaders for behaviour-change interventions. Methods: Healthcare professionals working in a single UK mental health NHS Foundation Trust were randomly allocated to one of two questionnaires. The first, slightly longer questionnaire, asked for multiple nominations of opinion leaders, with specific information about the nature of the relationship with each nominee. The second, shorter version, asked simply for a list of named “champions” but no more additional information. We compared, using Chi Square statistics, both the questionnaire response rates and the number of health professionals likely to be influenced by the opinion leaders (i.e. the “coverage” rates) for both questionnaire conditions. Results: Both questionnaire versions had low response rates: only 15% of health professionals named colleagues in the longer questionnaire and 13% in the shorter version. The opinion leaders identified by both methods had a low number of contacts (range of coverage, 2–6 each). There were no significant differences in response rates or coverage between the two identification methods. Conclusions: The low response and population coverage rates for both questionnaire versions suggest that alternative methods of identifying opinion leaders for implementation studies may be more effective. Future research should seek to identify and evaluate alternative, non-questionnaire based, methods of identifying opinion leaders in order to maximise their potential in organisational behaviour change interventions

Exploring the feasibility of Conjoint Analysis as a tool for prioritizing innovations for implementation

Background: In an era of scarce and competing priorities for implementation, choosing what to implement is a key decision point for many behavioural change projects. The values and attitudes of the professionals and managers involved inevitably impact the priority attached to decision options. Reliably capturing such values is challenging. Methods: This paper presents an approach for capturing and incorporating professional values into the prioritization of healthcare innovations being considered for adoption. Conjoint Analysis (CA) was used in a single UK Primary Care Trust to measure the priorities of healthcare professionals working with women with postnatal depression. Rating-based CA data was gathered using a questionnaire and then mapped onto 12 interventions being considered as a means of improving the management of postnatal depression. Results: The ‘impact on patient care’ and the ‘quality of supporting evidence’ associated with the potential innovations were the most influential in shaping priorities. Professionals were least influenced by whether an innovation was an existing national or local priority, or whether current practice in the Trust was meeting minimum standards. Ranking the 12 innovations by the preferences of potential adopters revealed ‘guided self help’ was the top priority for implementation and ‘screening questions for post natal depression’ the least. When other factors were considered (such as the presence of routine data or planned implementation activity elsewhere in the Trust), the project team chose to combine the eight related treatments and implement these as a single innovation referred to as ‘psychological therapies’. Conclusions: Using Conjoint Analysis to prioritise potential innovation implementation options is a feasible means of capturing the utility of stakeholders and thus increasing the chances of an innovation being adopted. There are some practical barriers to overcome such as increasing response rates to conjoint surveys before routine and unevaluated use of this technique should be considered

Use of a time series design to test effectiveness of a theory-based intervention targeting adherence of health professionals to a clinical guideline

Objectives. The aim of this study was to test the effectiveness of a theory of planned behaviour intervention to increase adherence of community mental health professionals to a national suicide prevention guideline. Design. Routinely collected audit adherence data from an intervention and control site were collected and analysed using time series analysis to test whether the intervention significantly increased adherence. The effects of a local and national event on adherence were also examined. Method. A Theory of Planned Behaviour (TPB) questionnaire, developed from interview findings, was administered to the health professionals. Subjective norms were found to be the most significant predictor of intention to adhere to the guideline, and were targeted with an interactive educational intervention. Time series analysis applied to routinely collected audit adherence data was used to test intervention effectiveness. Results. The TPB accounted for 58 of the variance in intention to adhere, with subjective norms the only significant predictor. The intervention did not significantly increase adherence; however, the national and local events were found to have significantly increased adherence. Conclusions. The TPB was a useful framework for exploring barriers to adherence; however, this did not translate into an effective intervention. Future research should seek collaboration with local experts, and use this information in combination with the TPB, to develop interventions. Collaborative research with experts in pedagogy may also help to develop more effective interventions, particularly education-based interventions that require adult learning

Evaluating change in a pressured healthcare system: a cross-sectional study of implementation outcomes using routine data indicators and proxies

Abstract Background Implementation evaluation should focus on implementation success, guided by theories and frameworks. With high staff vacancies in the health services, it is important to consider pragmatic methods of data collection for implementation evaluation. This paper presents a cross-sectional rapid evaluation of a handheld medical device designed for remote examinations, piloted in Northern England. By using downloaded device data and administrative records mapped to domains from the implementation outcomes framework, this evaluation offers a pragmatic example of assessing implementation success. Methods The pilot design was pragmatic: sites volunteered, decided which services to use the device in, and launched when ready. The pilot and evaluation together lasted 1 year. Data was downloaded from the devices, and administrative records for the pilot accessed. Variables were mapped to five of the implementation outcomes, after reviewing with the device manufacturer and pilot team to assess robustness. Results N=352 care episodes were recorded using the device with 223 patients. Out of 19 sites ‘signed up’ to the pilot, 5 launched and delivered 10 of 35 proposed projects: a site and project adoption rate of 26 and 29%, respectively. Six sites signed up to an extension period; three had launched and three had not during the original timelines, indicating some sustainability. Feasibility was high, with only one in seven care episodes needing to be repeated due to poor device quality or error (sound/audio/internet). Fidelity of device usage was low for two of the eight available device examinations. Device and staffing costs were high but potential cost savings were attributable to fewer in-person appointments. Conclusions Through using device and administrative data, this evaluation minimised burden on busy healthcare staff yet was still guided by an evaluation framework. Five out of the eight implementation outcomes were measured, including sustainability and costs. The findings give insight into implementation challenges, particularly around adoption. For future research, it is recommended to engage with staff to prioritise outcome measurements and to focus on meaningful interpretation of indicators

Maximizing the impact of systematic reviews in health care decision making: A systematic scoping review of knowledge-translation resources

Context: Barriers to the use of systematic reviews by policymakers may be overcome by resources that adapt and present the findings in formats more directly tailored to their needs. We performed a systematic scoping review to identify such knowledge-translation resources and evaluations of them. Methods: Resources were eligible for inclusion in this study if they were based exclusively or primarily on systematic reviews and were aimed at health care policymakers at the national or local level. Resources were identified by screening the websites of health technology assessment agencies and systematic review producers, supplemented by an email survey. Electronic databases and proceedings of the Cochrane Colloquium and HTA International were searched as well for published and unpublished evaluations of knowledge-translation resources. Resources were classified as summaries, overviews, or policy briefs using a previously published classification. Findings: Twenty knowledge-translation resources were identified, of which eleven were classified as summaries, six as overviews, and three as policy briefs. Resources added value to systematic reviews by, for example, evaluating their methodological quality or assessing the reliability of their conclusions or their generalizability to particular settings. The literature search found four published evaluation studies of knowledge-translation resources, and the screening of abstracts and contact with authors found three more unpublished studies. The majority of studies reported on the perceived usefulness of the service, although there were some examples of review-based resources being used to assist actual decision making. Conclusions: Systematic review producers provide a variety of resources to help policymakers, of which focused summaries are the most common. More evaluations of these resources are required to ensure users’ needs are being met, to demonstrate their impact, and to justify their funding