Technology for large-scale translation of clinical practice guidelines : a pilot study of the performance of a hybrid human and computer-assisted approach

Background: The construction of EBMPracticeNet, a national electronic point-of-care information platform in Belgium, was initiated in 2011 to optimize quality of care by promoting evidence-based decision-making. The project involved, among other tasks, the translation of 940 EBM Guidelines of Duodecim Medical Publications from English into Dutch and French. Considering the scale of the translation process, it was decided to make use of computer-aided translation performed by certificated translators with limited expertise in medical translation. Our consortium used a hybrid approach, involving a human translator supported by a translation memory (using SDL Trados Studio), terminology recognition (using SDL Multiterm termbases) from medical termbases and support from online machine translation. This has resulted in a validated translation memory which is now in use for the translation of new and updated guidelines. Objective: The objective of this study was to evaluate the performance of the hybrid human and computer-assisted approach in comparison with translation unsupported by translation memory and terminology recognition. A comparison was also made with the translation efficiency of an expert medical translator. Methods: We conducted a pilot trial in which two sets of 30 new and 30 updated guidelines were randomized to one of three groups. Comparable guidelines were translated (a) by certificated junior translators without medical specialization using the hybrid method (b) by an experienced medical translator without this support and (c) by the same junior translators without the support of the validated translation memory. A medical proofreader who was blinded for the translation procedure, evaluated the translated guidelines for acceptability and adequacy. Translation speed was measured by recording translation and post-editing time. The Human Translation Edit Rate was calculated as a metric to evaluate the quality of the translation. A further evaluation was made of translation acceptability and adequacy. Results: The average number of words per guideline was 1,195 and the mean total translation time was 100.2 min/1,000 words. No meaningful differences were found in the translation speed for new guidelines. The translation of updated guidelines was 59 min/1,000 words faster (95% CI 2-115; P=.044) in the computer-aided group. Revisions due to terminology accounted for one third of the overall revisions by the medical proofreader. Conclusions: Use of the hybrid human and computer-aided translation by a non-expert translator makes the translation of updates of clinical practice guidelines faster and cheaper because of the benefits of translation memory. For the translation of new guidelines there was no apparent benefit in comparison with the efficiency of translation unsupported by translation memory (whether by an expert or non-expert translator

Haemorrhagia post partum; an implementation study on the evidence-based guideline of the Dutch Society of Obstetrics and Gynaecology (NVOG) and the MOET (Managing Obstetric Emergencies and Trauma-course) instructions; the Fluxim study

Contains fulltext : 88435.pdf (publisher's version ) (Open Access)BACKGROUND: One of the most important causes of maternal mortality and severe morbidity worldwide is post partum haemorrhage (PPH). Factors as substandard care are frequently reported in the international literature and there are similar reports in the Netherlands. The incidence of PPH in the Dutch population is 5% containing 10.000 women a year. The introduction of an evidence-based guideline on PPH by the Dutch society of Obstetrics and Gynaecology (NVOG) and the initiation of the MOET course (Managing Obstetrics Emergencies and Trauma) did not lead to a reduction of PPH. This implies the possibility of an incomplete implementation of both the NVOG guideline and MOET-instructions. Therefore, the aim of this study is to develop and test a tailored strategy to implement both the NVOG guideline and MOET-instructions METHODS/DESIGN: One step in the development procedure is to evaluate the implementation of the guideline and MOET-instructions in the current care. Therefore measurement of the actual care will be performed in a representative sample of 20 hospitals. This will be done by prospective observation of the third stage of labour of 320 women with a high risk of PPH using quality indicators extracted from the NVOG guideline and MOET instructions. In the next step barriers and facilitators for guideline adherence will be analyzed by performance of semi structured interviews with 30 professionals and 10 patients, followed by a questionnaire study among all Dutch gynaecologists and midwives to quantify the barriers mentioned. Based on the outcomes, a tailored strategy to implement the NVOG guideline and MOET-instructions will be developed and tested in a feasibility study in 4 hospitals, including effect-, process- and cost evaluation. DISCUSSION: This study will provide insight into current Dutch practice, in particular to what extent the PPH guidelines of the NVOG and the MOET-instructions have been implemented in the actual care, and into the barriers and facilitators regarding guideline adherence. The knowledge of the feasibility study regarding the effects and costs of the tailored strategy and the experiences of the users can be used in countries with a relatively high incidence of PPH. TRIAL REGISTRATION: ClinicTrials.gov NCT00928863

The cost-effectiveness of quality improvement projects: a conceptual framework, checklist and online tool for considering the costs and consequences of implementation based quality improvement

In resource constrained systems, decision makers should be concerned with the efficiency of implementing improvement techniques and technologies. Accordingly, they should consider both the costs and effectiveness of implementation as well as the cost-effectiveness of the innovation to be implemented. An approach to doing this effectively is encapsulated in the ‘policy cost-effectiveness’ approach. This paper outlines some of the theoretical and practical challenges to assessing policy cost-effectiveness (the cost-effectiveness of implementation projects). A checklist and associated (freely available) online application are also presented to help services develop more cost-effective implementation strategies

Collaborating with front-line healthcare professionals: the clinical and cost effectiveness of a theory based approach to the implementation of a national guideline

Background Clinical guidelines are an integral part of healthcare. Whilst much progress has been made in ensuring that guidelines are well developed and disseminated, the gap between routine clinical practice and current guidelines often remains wide. A key reason for this gap is that implementation of guidelines typically requires a change in the behaviour of healthcare professionals – but the behaviour change component is often overlooked. We adopted the Theoretical Domains Framework Implementation (TDFI) approach for supporting behaviour change required for the uptake of a national patient safety guideline to reduce the risk of feeding through misplaced nasogastric tubes. Methods The TDFI approach was used in a pre-post study in three NHS hospitals with a fourth acting as a control (with usual care and no TDFI). The target behavior identified for change was to increase the use of pH testing as the first line method for checking the position of a nasogastric tube. Repeat audits were undertaken in each hospital following intervention implementation. We used Zou’s modified Poisson regression approach with robust standard errors to estimate risk ratios for the use of pH testing. The projected return on investment (ROI) was also calculated. Results Following intervention implementation, the use of pH first line increased significantly across intervention hospitals [risk ratio (95% CI) ranged from 3.1 (1.14 to8.43) p < .05, to 8.14 (3.06 to21.67) p < .001] compared to the control hospital, which remained unchanged [risk ratio (CI) = .77 (.47-1.26) p = .296]. The estimated savings and costs in the first year were £2.56 million and £1.41 respectively, giving an ROI of 82%, and this was projected to increase to 270% over five years. Conclusion The TDFI approach improved the uptake of a patient safety guideline across three hospitals. The TDFI approach is clinically and cost effective in comparison to the usual practice

Designing theoretically-informed implementation interventions

Clinical and health services research is continually producing new findings that may contribute to effective and efficient patient care. However, the transfer of research findings into practice is unpredictable and can be a slow and haphazard process. Ideally, the choice of implementation strategies would be based upon evidence from randomised controlled trials or systematic reviews of a given implementation strategy. Unfortunately, reviews of implementation strategies consistently report effectiveness some, but not all of the time; possible causes of this variation are seldom reported or measured by the investigators in the original studies. Thus, any attempts to extrapolate from study settings to the real world are hampered by a lack of understanding of the effects of key elements of individuals, interventions, and the settings in which they were trialled. The explicit use of theory offers a way of addressing these issues and has a number of advantages, such as providing: a generalisable framework within which to represent the dimensions that implementation studies address, a process by which to inform the development and delivery of interventions, a guide when evaluating, and a way to allow for an exploration of potential causal mechanisms. However, the use of theory in designing implementation interventions is methodologically challenging for a number of reasons, including choosing between theories and faithfully translating theoretical constructs into interventions. The explicit use of theory offers potential advantages in terms of facilitating a better understanding of the generalisability and replicability of implementation interventions. However, this is a relatively unexplored methodological area

Implementation of an evidence-based guideline on fluid resuscitation: lessons learnt for future guidelines

There is little experience with the nationwide implementation of an evidence-based pediatric guideline on first-choice fluid for resuscitation in hypovolemia. We investigated fluid prescribing behavior at (1) guideline development, (2) after guideline development, and (3) after active implementation and identified potential barriers and facilitators for guideline implementation. In order to minimize costs and to optimize implementation effect, we continuously developed and adjusted implementation strategies according to identified barriers. Implementation success was evaluated using questionnaires, pharmaceutical data, and data from medical records. The most remarkable change occurred after guideline development and dissemination: Normal saline use by neonatologists increased from 22-89% to 100% and by pediatric intensivists from 43-71% to 88-100%, and synthetic colloid use by pediatric intensivists declined from 29-43% to 0-13% with a reduction in albumin use by neonatologists from 11-44% to 0%. After active guideline implementation, most of specialist's management behavior was according to the guideline. Stakeholders involved in the developmental process are of great importance in disseminating recommendations before active implementation. Therefore, to successfully implement guidelines and reduce costs of active implementation, any guideline development should consider implementation right from the beginning. Implementation strategies should target identified barriers and will therefore always be guideline specifi

High workload and job stress are associated with lower practice performance in general practice: an observational study in 239 general practices in the Netherlands

Contains fulltext : 80493.pdf (publisher's version ) (Open Access)BACKGROUND: The impact of high physician workload and job stress on quality and outcomes of healthcare delivery is not clear. Our study explored whether high workload and job stress were associated with lower performance in general practices in the Netherlands. METHODS: Secondary analysis of data from 239 general practices, collected in practice visits between 2003 to 2006 in the Netherlands using a comprehensive set of measures of practice management. Data were collected by a practice visitor, a trained non-physician observer using patients questionnaires, doctors and staff. For this study we selected five measures of practice performance as outcomes and six measures of GP workload and job stress as predictors. A total of 79 indicators were used out of the 303 available indicators. Random coefficient regression models were applied to examine associations. RESULTS AND DISCUSSION: Workload and job stress are associated with practice performance.Workload: Working more hours as a GP was associated with more positive patient experiences of accessibility and availability (b = 0.16). After list size adjustment, practices with more GP-time per patient scored higher on GP care (b = 0.45). When GPs provided more than 20 hours per week per 1000 patients, patients scored over 80% on the Europep questionnaire for quality of GP care.Job stress: High GP job stress was associated with lower accessibility and availability (b = 0.21) and insufficient practice management (b = 0.25). Higher GP commitment and more satisfaction with the job was associated with more prevention and disease management (b = 0.35). CONCLUSION: Providing more time in the practice, and more time per patient and experiencing less job stress are all associated with perceptions by patients of better care and better practice performance. Workload and job stress should be assessed by using list size adjusted data in order to realise better quality of care. Organisational development using this kind of data feedback could benefit both patients and GP

From theory to 'measurement' in complex interventions: methodological lessons from the development of an e-health normalisation instrument

&lt;b&gt;Background&lt;/b&gt; Although empirical and theoretical understanding of processes of implementation in health care is advancing, translation of theory into structured measures that capture the complex interplay between interventions, individuals and context remain limited. This paper aimed to (1) describe the process and outcome of a project to develop a theory-based instrument for measuring implementation processes relating to e-health interventions; and (2) identify key issues and methodological challenges for advancing work in this field.&lt;p&gt;&lt;/p&gt; &lt;b&gt;Methods&lt;/b&gt; A 30-item instrument (Technology Adoption Readiness Scale (TARS)) for measuring normalisation processes in the context of e-health service interventions was developed on the basis on Normalization Process Theory (NPT). NPT focuses on how new practices become routinely embedded within social contexts. The instrument was pre-tested in two health care settings in which e-health (electronic facilitation of healthcare decision-making and practice) was used by health care professionals.&lt;p&gt;&lt;/p&gt; &lt;b&gt;Results&lt;/b&gt; The developed instrument was pre-tested in two professional samples (N = 46; N = 231). Ratings of items representing normalisation 'processes' were significantly related to staff members' perceptions of whether or not e-health had become 'routine'. Key methodological challenges are discussed in relation to: translating multi-component theoretical constructs into simple questions; developing and choosing appropriate outcome measures; conducting multiple-stakeholder assessments; instrument and question framing; and more general issues for instrument development in practice contexts.&lt;p&gt;&lt;/p&gt; &lt;b&gt;Conclusions&lt;/b&gt; To develop theory-derived measures of implementation process for progressing research in this field, four key recommendations are made relating to (1) greater attention to underlying theoretical assumptions and extent of translation work required; (2) the need for appropriate but flexible approaches to outcomes measurement; (3) representation of multiple perspectives and collaborative nature of work; and (4) emphasis on generic measurement approaches that can be flexibly tailored to particular contexts of study

Out-of-hours care in western countries: assessment of different organizational models

Contains fulltext : 81655.pdf (publisher's version ) (Open Access)BACKGROUND: Internationally, different organizational models are used for providing out-of-hours care. The aim of this study was to assess prevailing models in order to identify their potential strengths and weaknesses. METHODS: An international web-based survey was done in 2007 in a sample of purposefully selected key informants from 25 western countries. The questions concerned prevailing organizational models for out-of-hours care, the most dominant model in each country, perceived weaknesses, and national plans for changes in out-of-hours care. RESULTS: A total of 71 key informants from 25 countries provided answers. In most countries several different models existed alongside each other. The Accident and Emergency department was the organizational model most frequently used. Perceived weaknesses of this model concerned the coordination and continuity of care, its efficiency and accessibility. In about a third of the countries, the rota group was the most dominant organizational model for out-of-hours care. A perceived weakness of this model was lowered job satisfaction of physicians. The GP cooperative existed in a majority of the participating countries; no weaknesses were mentioned with respect to this model. Most of the countries had plans to change the out-of-hours care, mainly toward large scale organizations. CONCLUSION: GP cooperatives combine size of scale advantages with organizational features of strong primary care, such as high accessibility, continuity and coordination of care. While specific patients require other organizational models, the co-existence of different organizational models for out-of-hours care in a country may be less efficient for health systems

Primary care nurses struggle with lifestyle counseling in diabetes care: a qualitative analysis

Contains fulltext : 89605.pdf (publisher's version ) (Open Access)BACKGROUND: Patient outcomes are poorly affected by lifestyle advice in general practice. Promoting lifestyle behavior change require that nurses shift from simple advice giving to a more counseling-based approach. The current study examines which barriers nurses encounter in lifestyle counseling to patients with type 2 diabetes. Based on this information we will develop an implementation strategy to improve lifestyle behavior change in general practice. METHOD: In a qualitative semi-structured study, twelve in-depth interviews took place with nurses in Dutch general practices involved in diabetes care. Specific barriers in counseling patients with type 2 diabetes about diet, physical activity, and smoking cessation were addressed. The nurses were invited to reflect on barriers at the patient and practice levels, but mainly on their own roles as counselors. All interviews were audio-recorded and transcribed. The data were analyzed with the aid of a predetermined framework. RESULTS: Nurses felt most barriers on the level of the patient; patients had limited knowledge of a healthy lifestyle and limited insight into their own behavior, and they lacked the motivation to modify their lifestyles or the discipline to maintain an improved lifestyle. Furthermore, nurses reported lack of counseling skills and insufficient time as barriers in effective lifestyle counseling. CONCLUSIONS: The traditional health education approach is still predominant in primary care of patients with type 2 diabetes. An implementation strategy based on motivational interviewing can help to overcome 'jumping ahead of the patient' and promotes skills in lifestyle behavioral change. We will train our nurses in agenda setting to structure the consultation based on prioritizing the behavior change and will help them to develop social maps that contain information on local exercise programs