NorthStar, a support tool for the design and evaluation of quality improvement interventions in healthcare

Background: The Research-Based Education and Quality Improvement (ReBEQI) European partnership aims to establish a framework and provide practical tools for the selection, implementation, and evaluation of quality improvement (QI) interventions. We describe the development and preliminary evaluation of the software tool NorthStar, a major product of the ReBEQI project. Methods: We focused the content of NorthStar on the design and evaluation of QI interventions. A lead individual from the ReBEQI group drafted each section, and at least two other group members reviewed it. The content is based on published literature, as well as material developed by the ReBEQI group. We developed the software in both a Microsoft Windows HTML help system version and a web-based version. In a preliminary evaluation, we surveyed 33 potential users about the acceptability and perceived utility of NorthStar. Results: NorthStar consists of 18 sections covering the design and evaluation of QI interventions. The major focus of the intervention design sections is on how to identify determinants of practice (factors affecting practice patterns), while the major focus of the intervention evaluation sections is on how to design a cluster randomised trial. The two versions of the software can be transferred by email or CD, and are available for download from the internet. The software offers easy navigation and various functions to access the content. Potential users (55% response rate) reported above-moderate levels of confidence in carrying out QI research related tasks if using NorthStar, particularly when developing a protocol for a cluster randomised trial Conclusion: NorthStar is an integrated, accessible, practical, and acceptable tool to assist developers and evaluators of QI interventions

Technology for large-scale translation of clinical practice guidelines : a pilot study of the performance of a hybrid human and computer-assisted approach

Background: The construction of EBMPracticeNet, a national electronic point-of-care information platform in Belgium, was initiated in 2011 to optimize quality of care by promoting evidence-based decision-making. The project involved, among other tasks, the translation of 940 EBM Guidelines of Duodecim Medical Publications from English into Dutch and French. Considering the scale of the translation process, it was decided to make use of computer-aided translation performed by certificated translators with limited expertise in medical translation. Our consortium used a hybrid approach, involving a human translator supported by a translation memory (using SDL Trados Studio), terminology recognition (using SDL Multiterm termbases) from medical termbases and support from online machine translation. This has resulted in a validated translation memory which is now in use for the translation of new and updated guidelines. Objective: The objective of this study was to evaluate the performance of the hybrid human and computer-assisted approach in comparison with translation unsupported by translation memory and terminology recognition. A comparison was also made with the translation efficiency of an expert medical translator. Methods: We conducted a pilot trial in which two sets of 30 new and 30 updated guidelines were randomized to one of three groups. Comparable guidelines were translated (a) by certificated junior translators without medical specialization using the hybrid method (b) by an experienced medical translator without this support and (c) by the same junior translators without the support of the validated translation memory. A medical proofreader who was blinded for the translation procedure, evaluated the translated guidelines for acceptability and adequacy. Translation speed was measured by recording translation and post-editing time. The Human Translation Edit Rate was calculated as a metric to evaluate the quality of the translation. A further evaluation was made of translation acceptability and adequacy. Results: The average number of words per guideline was 1,195 and the mean total translation time was 100.2 min/1,000 words. No meaningful differences were found in the translation speed for new guidelines. The translation of updated guidelines was 59 min/1,000 words faster (95% CI 2-115; P=.044) in the computer-aided group. Revisions due to terminology accounted for one third of the overall revisions by the medical proofreader. Conclusions: Use of the hybrid human and computer-aided translation by a non-expert translator makes the translation of updates of clinical practice guidelines faster and cheaper because of the benefits of translation memory. For the translation of new guidelines there was no apparent benefit in comparison with the efficiency of translation unsupported by translation memory (whether by an expert or non-expert translator

Designing theoretically-informed implementation interventions

Clinical and health services research is continually producing new findings that may contribute to effective and efficient patient care. However, the transfer of research findings into practice is unpredictable and can be a slow and haphazard process. Ideally, the choice of implementation strategies would be based upon evidence from randomised controlled trials or systematic reviews of a given implementation strategy. Unfortunately, reviews of implementation strategies consistently report effectiveness some, but not all of the time; possible causes of this variation are seldom reported or measured by the investigators in the original studies. Thus, any attempts to extrapolate from study settings to the real world are hampered by a lack of understanding of the effects of key elements of individuals, interventions, and the settings in which they were trialled. The explicit use of theory offers a way of addressing these issues and has a number of advantages, such as providing: a generalisable framework within which to represent the dimensions that implementation studies address, a process by which to inform the development and delivery of interventions, a guide when evaluating, and a way to allow for an exploration of potential causal mechanisms. However, the use of theory in designing implementation interventions is methodologically challenging for a number of reasons, including choosing between theories and faithfully translating theoretical constructs into interventions. The explicit use of theory offers potential advantages in terms of facilitating a better understanding of the generalisability and replicability of implementation interventions. However, this is a relatively unexplored methodological area

Collaborating with front-line healthcare professionals: the clinical and cost effectiveness of a theory based approach to the implementation of a national guideline

Background Clinical guidelines are an integral part of healthcare. Whilst much progress has been made in ensuring that guidelines are well developed and disseminated, the gap between routine clinical practice and current guidelines often remains wide. A key reason for this gap is that implementation of guidelines typically requires a change in the behaviour of healthcare professionals – but the behaviour change component is often overlooked. We adopted the Theoretical Domains Framework Implementation (TDFI) approach for supporting behaviour change required for the uptake of a national patient safety guideline to reduce the risk of feeding through misplaced nasogastric tubes. Methods The TDFI approach was used in a pre-post study in three NHS hospitals with a fourth acting as a control (with usual care and no TDFI). The target behavior identified for change was to increase the use of pH testing as the first line method for checking the position of a nasogastric tube. Repeat audits were undertaken in each hospital following intervention implementation. We used Zou’s modified Poisson regression approach with robust standard errors to estimate risk ratios for the use of pH testing. The projected return on investment (ROI) was also calculated. Results Following intervention implementation, the use of pH first line increased significantly across intervention hospitals [risk ratio (95% CI) ranged from 3.1 (1.14 to8.43) p < .05, to 8.14 (3.06 to21.67) p < .001] compared to the control hospital, which remained unchanged [risk ratio (CI) = .77 (.47-1.26) p = .296]. The estimated savings and costs in the first year were £2.56 million and £1.41 respectively, giving an ROI of 82%, and this was projected to increase to 270% over five years. Conclusion The TDFI approach improved the uptake of a patient safety guideline across three hospitals. The TDFI approach is clinically and cost effective in comparison to the usual practice

From theory to 'measurement' in complex interventions: methodological lessons from the development of an e-health normalisation instrument

&lt;b&gt;Background&lt;/b&gt; Although empirical and theoretical understanding of processes of implementation in health care is advancing, translation of theory into structured measures that capture the complex interplay between interventions, individuals and context remain limited. This paper aimed to (1) describe the process and outcome of a project to develop a theory-based instrument for measuring implementation processes relating to e-health interventions; and (2) identify key issues and methodological challenges for advancing work in this field.&lt;p&gt;&lt;/p&gt; &lt;b&gt;Methods&lt;/b&gt; A 30-item instrument (Technology Adoption Readiness Scale (TARS)) for measuring normalisation processes in the context of e-health service interventions was developed on the basis on Normalization Process Theory (NPT). NPT focuses on how new practices become routinely embedded within social contexts. The instrument was pre-tested in two health care settings in which e-health (electronic facilitation of healthcare decision-making and practice) was used by health care professionals.&lt;p&gt;&lt;/p&gt; &lt;b&gt;Results&lt;/b&gt; The developed instrument was pre-tested in two professional samples (N = 46; N = 231). Ratings of items representing normalisation 'processes' were significantly related to staff members' perceptions of whether or not e-health had become 'routine'. Key methodological challenges are discussed in relation to: translating multi-component theoretical constructs into simple questions; developing and choosing appropriate outcome measures; conducting multiple-stakeholder assessments; instrument and question framing; and more general issues for instrument development in practice contexts.&lt;p&gt;&lt;/p&gt; &lt;b&gt;Conclusions&lt;/b&gt; To develop theory-derived measures of implementation process for progressing research in this field, four key recommendations are made relating to (1) greater attention to underlying theoretical assumptions and extent of translation work required; (2) the need for appropriate but flexible approaches to outcomes measurement; (3) representation of multiple perspectives and collaborative nature of work; and (4) emphasis on generic measurement approaches that can be flexibly tailored to particular contexts of study

The cost-effectiveness of quality improvement projects: a conceptual framework, checklist and online tool for considering the costs and consequences of implementation based quality improvement

In resource constrained systems, decision makers should be concerned with the efficiency of implementing improvement techniques and technologies. Accordingly, they should consider both the costs and effectiveness of implementation as well as the cost-effectiveness of the innovation to be implemented. An approach to doing this effectively is encapsulated in the ‘policy cost-effectiveness’ approach. This paper outlines some of the theoretical and practical challenges to assessing policy cost-effectiveness (the cost-effectiveness of implementation projects). A checklist and associated (freely available) online application are also presented to help services develop more cost-effective implementation strategies

FIRE (facilitating implementation of research evidence) : a study protocol

Research evidence underpins best practice, but is not always used in healthcare. The Promoting Action on Research Implementation in Health Services (PARIHS) framework suggests that the nature of evidence, the context in which it is used, and whether those trying to use evidence are helped (or facilitated) affect the use of evidence. Urinary incontinence has a major effect on quality of life of older people, has a high prevalence, and is a key priority within European health and social care policy. Improving continence care has the potential to improve the quality of life for older people and reduce the costs associated with providing incontinence aids

Ethical issues in implementation research: a discussion of the problems in achieving informed consent

Background: Improved quality of care is a policy objective of health care systems around the world. Implementation research is the scientific study of methods to promote the systematic uptake of clinical research findings into routine clinical practice, and hence to reduce inappropriate care. It includes the study of influences on healthcare professionals' behaviour and methods to enable them to use research findings more effectively. Cluster randomized trials represent the optimal design for evaluating the effectiveness of implementation strategies. Various codes of medical ethics, such as the Nuremberg Code and the Declaration of Helsinki inform medical research, but their relevance to cluster randomised trials in implementation research is unclear. This paper discusses the applicability of various ethical codes to obtaining consent in cluster trials in implementation research. Discussion: The appropriate application of biomedical codes to implementation research is not obvious. Discussion of the nature and practice of informed consent in implementation research cluster trials must consider the levels at which consent can be sought, and for what purpose it can be sought. The level at which an intervention is delivered can render the idea of patient level consent meaningless. Careful consideration of the ownership of information, and rights of access to and exploitation of data is required. For health care professionals and organizations, there is a balance between clinical freedom and responsibility to participate in research. Summary: While ethical justification for clinical trials relies heavily on individual consent, for implementation research aspects of distributive justice, economics, and political philosophy underlie the debate. Societies may need to trade off decisions on the choice between individualized consent and valid implementation research. We suggest that social sciences codes could usefully inform the consideration of implementation research by members of Research Ethics Committees

Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science

Abstract Background Many interventions found to be effective in health services research studies fail to translate into meaningful patient care outcomes across multiple contexts. Health services researchers recognize the need to evaluate not only summative outcomes but also formative outcomes to assess the extent to which implementation is effective in a specific setting, prolongs sustainability, and promotes dissemination into other settings. Many implementation theories have been published to help promote effective implementation. However, they overlap considerably in the constructs included in individual theories, and a comparison of theories reveals that each is missing important constructs included in other theories. In addition, terminology and definitions are not consistent across theories. We describe the Consolidated Framework For Implementation Research (CFIR) that offers an overarching typology to promote implementation theory development and verification about what works where and why across multiple contexts. Methods We used a snowball sampling approach to identify published theories that were evaluated to identify constructs based on strength of conceptual or empirical support for influence on implementation, consistency in definitions, alignment with our own findings, and potential for measurement. We combined constructs across published theories that had different labels but were redundant or overlapping in definition, and we parsed apart constructs that conflated underlying concepts. Results The CFIR is composed of five major domains: intervention characteristics, outer setting, inner setting, characteristics of the individuals involved, and the process of implementation. Eight constructs were identified related to the intervention (e.g., evidence strength and quality), four constructs were identified related to outer setting (e.g., patient needs and resources), 12 constructs were identified related to inner setting (e.g., culture, leadership engagement), five constructs were identified related to individual characteristics, and eight constructs were identified related to process (e.g., plan, evaluate, and reflect). We present explicit definitions for each construct. Conclusion The CFIR provides a pragmatic structure for approaching complex, interacting, multi-level, and transient states of constructs in the real world by embracing, consolidating, and unifying key constructs from published implementation theories. It can be used to guide formative evaluations and build the implementation knowledge base across multiple studies and settings.http://deepblue.lib.umich.edu/bitstream/2027.42/78272/1/1748-5908-4-50.xmlhttp://deepblue.lib.umich.edu/bitstream/2027.42/78272/2/1748-5908-4-50-S1.PDFhttp://deepblue.lib.umich.edu/bitstream/2027.42/78272/3/1748-5908-4-50-S3.PDFhttp://deepblue.lib.umich.edu/bitstream/2027.42/78272/4/1748-5908-4-50-S4.PDFhttp://deepblue.lib.umich.edu/bitstream/2027.42/78272/5/1748-5908-4-50.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/78272/6/1748-5908-4-50-S2.PDFPeer Reviewe

Undergraduate teaching in geriatric medicine: mapping the British Geriatrics Society undergraduate curriculum to Tomorrow's Doctors 2009

Introduction: in 2008, the British Geriatrics Society (BGS) developed the Recommended Undergraduate Curriculum in Geriatric Medicine. This was subsequently mapped to the second edition of Tomorrows' Doctors (TD2, 2003). Following the publication of the third edition of Tomorrow's Doctors in 2009 (TD3), the mapping exercise was repeated to verify the extent to which the updated General Medical Council recommendations supported teaching in ageing and geriatric medicine. Method: we analysed TD3 and identified 48 aspects of its general guidance that were relevant to the teaching of medicine for older people. We then mapped these to the 2009 BGS curriculum. Results: the BGS curriculum was supported in full by TD3. However, learning outcomes relating to the interpretation and conduct of research in TD3 had no corresponding outcomes in the BGS curriculum. Conclusion: the BGS curriculum for medical undergraduates continues to provide a specific and complete list of learning objectives, all of which could help to operationalise the general statements made in TD3 with relation to ageing and geriatric medicine. Learning outcomes in research in frail older patients have been added following this mapping exercise