What do we know about how to do audit and feedback? Pitfalls in applying evidence from a systematic review

BACKGROUND: Improving the quality of health care requires a range of evidence-based activities. Audit and feedback is commonly used as a quality improvement tool in the UK National Health Service [NHS]. We set out to assess whether current guidance and systematic review evidence can sufficiently inform practical decisions about how to use audit and feedback to improve quality of care. METHODS: We selected an important chronic disease encountered in primary care: diabetes mellitus. We identified recommendations from National Institute for Clinical Excellence (NICE) guidance on conducting audit and generated questions which would be relevant to any attempt to operationalise audit and feedback in a healthcare service setting. We explored the extent to which a systematic review of audit and feedback could provide practical guidance about whether audit and feedback should be used to improve quality of diabetes care and, if so, how audit and feedback could be optimised. RESULTS: National guidance suggests the importance of securing the right organisational conditions and processes. Review evidence suggests that audit and feedback can be effective in changing healthcare professional practice. However, the available evidence says relatively little about the detail of how to use audit and feedback most efficiently. CONCLUSION: Audit and feedback will continue to be an unreliable approach to quality improvement until we learn how and when it works best. Conceptualising audit and feedback within a theoretical framework offers a way forward

A pragmatic randomised trial of stretching before and after physical activity to prevent injury and soreness

OBJECTIVE: To determine the effects of stretching before and after physical activity on risks of injury and soreness in a community population. DESIGN: Internet-based pragmatic randomised trial conducted between January 2008 and January 2009. SETTING: International. PARTICIPANTS: A total of 2377 adults who regularly participated in physical activity. INTERVENTIONS: Participants in the stretch group were asked to perform 30 s static stretches of seven lower limb and trunk muscle groups before and after physical activity for 12 weeks. Participants in the control group were asked not to stretch. MAIN OUTCOME MEASUREMENTS: Participants provided weekly on-line reports of outcomes over 12 weeks. Primary outcomes were any injury to the lower limb or back, and bothersome soreness of the legs, buttocks or back. Injury to muscles, ligaments and tendons was a secondary outcome. RESULTS: Stretching did not produce clinically important or statistically significant reductions in all-injury risk (HR=0.97, 95% CI 0.84 to 1.13), but did reduce the risk of experiencing bothersome soreness (mean risk of bothersome soreness in a week was 24.6% in the stretch group and 32.3% in the control group; OR=0.69, 95% CI 0.59 to 0.82). Stretching reduced the risk of injuries to muscles, ligaments and tendons (incidence rate of 0.66 injuries per person-year in the stretch group and 0.88 injuries per person-year in the control group; HR=0.75, 95% CI 0.59 to 0.96). CONCLUSION: Stretching before and after physical activity does not appreciably reduce all-injury risk but probably reduces the risk of some injuries, and does reduce the risk of bothersome soreness. TRIAL REGISTRATION: anzctr.org.au 12608000044325

Designing theoretically-informed implementation interventions

Clinical and health services research is continually producing new findings that may contribute to effective and efficient patient care. However, the transfer of research findings into practice is unpredictable and can be a slow and haphazard process. Ideally, the choice of implementation strategies would be based upon evidence from randomised controlled trials or systematic reviews of a given implementation strategy. Unfortunately, reviews of implementation strategies consistently report effectiveness some, but not all of the time; possible causes of this variation are seldom reported or measured by the investigators in the original studies. Thus, any attempts to extrapolate from study settings to the real world are hampered by a lack of understanding of the effects of key elements of individuals, interventions, and the settings in which they were trialled. The explicit use of theory offers a way of addressing these issues and has a number of advantages, such as providing: a generalisable framework within which to represent the dimensions that implementation studies address, a process by which to inform the development and delivery of interventions, a guide when evaluating, and a way to allow for an exploration of potential causal mechanisms. However, the use of theory in designing implementation interventions is methodologically challenging for a number of reasons, including choosing between theories and faithfully translating theoretical constructs into interventions. The explicit use of theory offers potential advantages in terms of facilitating a better understanding of the generalisability and replicability of implementation interventions. However, this is a relatively unexplored methodological area

Data for improvement and clinical excellence: protocol for an audit with feedback intervention in long-term care

<p>Abstract</p> <p>Background</p> <p>There is considerable evidence about the effectiveness of audit coupled with feedback, although few audit with feedback interventions have been conducted in long-term care (LTC) settings to date. In general, the effects have been found to be modest at best, although in settings where there has been little history of audit and feedback, the effects may be greater, at least initially. The primary purpose of the Data for Improvement and Clinical Excellence (DICE) Long-Term Care project is to assess the effects of an audit with feedback intervention delivered monthly over 13 months in four LTC facilities. The research questions we addressed are:</p> <p indent="1">1. What effects do feedback reports have on processes and outcomes over time?</p> <p indent="1">2. How do different provider groups in LTC and home care respond to feedback reports based on data targeted at improving quality of care?</p> <p>Methods/design</p> <p>The research team conducting this study comprises researchers and decision makers in continuing care in the province of Alberta, Canada. The intervention consists of monthly feedback reports in nine LTC units in four facilities in Edmonton, Alberta. Data for the feedback reports comes from the Resident Assessment Instrument Minimum Data Set (RAI) version 2.0, a standardized instrument mandated for use in LTC facilities throughout Alberta. Feedback reports consist of one page, front and back, presenting both graphic and textual information. Reports are delivered to all staff working in the four LTC facilities. The primary evaluation uses a controlled interrupted time series design both adjusted and unadjusted for covariates. The concurrent process evaluation uses observation and self-report to assess uptake of the feedback reports. Following the project phase described in this protocol, a similar intervention will be conducted in home care settings in Alberta. Depending on project findings, if they are judged useful by decision makers participating in this research team, we plan dissemination and spread of the feedback report approach throughout Alberta.</p

Effective Feedback to Improve Primary Care Prescribing Safety (EFIPPS) a pragmatic three-arm cluster randomised trial:designing the intervention (ClinicalTrials.gov registration NCT01602705)

Peer reviewedPublisher PD

Measuring physiotherapy performance in patients with osteoarthritis of the knee: A prospective study

<p>Abstract</p> <p>Background</p> <p>Patients with knee osteoarthritis [OA] are commonly treated by physiotherapists in primary care. Measuring physiotherapy performance is important before developing strategies to improve quality. The purpose of this study was to measure physiotherapy performance in patients with knee OA by comparing clinical practice to evidence from systematic reviews.</p> <p>Methods</p> <p>We developed a data-collection form and invited all private practitioners in Norway [n = 2798] to prospectively collect data on the management of one patient with knee OA through 12 treatment session. Actual practice was compared to findings from an overview of systematic reviews summarising the effect of physiotherapy interventions for knee OA.</p> <p>Results</p> <p>A total of 297 physiotherapists reported their management for patients with knee OA. Exercise was the most common treatment used, provided by 98% of the physiotherapists. There is evidence of high quality that exercise reduces pain and improves function in patients with knee OA. Thirty-five percent of physiotherapists used acupuncture, low-level laser therapy or transcutaneous electrical nerve stimulation. There is evidence of moderate quality that these treatments reduce pain in knee OA. Patient education, supported by moderate quality evidence for improving psychological outcomes, was provided by 68%. Physiotherapists used a median of four different treatment modalities for each patient. They offered many treatment modalities based on evidence of low quality or without evidence from systematic reviews, e.g. traction and mobilisation, massage and stretching.</p> <p>Conclusion</p> <p>Exercise was used in almost all treatment sessions in the management of knee OA. This practice is desirable since it is supported by high quality evidence. Physiotherapists also provide several other treatment modalities based on evidence of moderate or low quality, or no evidence from systematic reviews. Ways to promote high quality evidence into physiotherapy practice should be identified and evaluated.</p

From the Trenches: A Cross-Sectional Study Applying the GRADE Tool in Systematic Reviews of Healthcare Interventions

Background: GRADE was developed to address shortcomings of tools to rate the quality of a body of evidence. While much has been published about GRADE, there are few empirical and systematic evaluations. Objective: To assess GRADE for systematic reviews (SRs) in terms of inter-rater agreement and identify areas of uncertainty. Design: Cross-sectional, descriptive study. Methods: We applied GRADE to three SRs (n = 48, 66, and 75 studies, respectively) with 29 comparisons and 12 outcomes overall. Two reviewers graded evidence independently for outcomes deemed clinically important a priori. Inter-rater reliability was assessed using kappas for four main domains (risk of bias, consistency, directness, and precision) and overall quality of evidence. Results: For the first review, reliability was: k = 0.41 for risk of bias; 0.84 consistency; 0.18 precision; and 0.44 overall quality. Kappa could not be calculated for directness as one rater assessed all items as direct; assessors agreed in 41 % of cases. For the second review reliability was: 0.37 consistency and 0.19 precision. Kappa could not be assessed for other items; assessors agreed in 33 % of cases for risk of bias; 100 % directness; and 58 % overall quality. For the third review, reliability was: 0.06 risk of bias; 0.79 consistency; 0.21 precision; and 0.18 overall quality. Assessors agreed in 100 % of cases for directness. Precision created the most uncertainty due to difficulties in identifying ‘‘optimal’ ’ information size and ‘‘clinica

Improving obstetric care in low-resource settings: implementation of facility-based maternal death reviews in five pilot hospitals in Senegal

<p>Abstract</p> <p>Background</p> <p>In sub-Saharan Africa, maternal and perinatal mortality and morbidity are major problems. Service availability and quality of care in health facilities are heterogeneous and most often inadequate. In resource-poor settings, the facility-based maternal death review or audit is one of the most promising strategies to improve health service performance. We aim to explore and describe health workers' perceptions of facility-based maternal death reviews and to identify barriers to and facilitators of the implementation of this approach in pilot health facilities of Senegal.</p> <p>Methods</p> <p>This study was conducted in five reference hospitals in Senegal with different characteristics. Data were collected from focus group discussions, participant observations of audit meetings, audit documents and interviews with the staff of the maternity unit. Data were analysed by means of both quantitative and qualitative approaches.</p> <p>Results</p> <p>Health professionals and service administrators were receptive and adhered relatively well to the process and the results of the audits, although some considered the situation destabilizing or even threatening. The main barriers to the implementation of maternal deaths reviews were: (1) bad quality of information in medical files; (2) non-participation of the head of department in the audit meetings; (3) lack of feedback to the staff who did not attend the audit meetings. The main facilitators were: (1) high level of professional qualifications or experience of the data collector; (2) involvement of the head of the maternity unit, acting as a moderator during the audit meetings; (3) participation of managers in the audit session to plan appropriate and realistic actions to prevent other maternal deaths.</p> <p>Conclusion</p> <p>The identification of the barriers to and the facilitators of the implementation of maternal death reviews is an essential step for the future adaptation of this method in countries with few resources. We recommend for future implementation of this method a prior enhancement of the perinatal information system and initial training of the members of the audit committee – particularly the data collector and the head of the maternity unit. Local leadership is essential to promote, initiate and monitor the audit process in the health facilities.</p

A hospital-site controlled intervention using audit and feedback to implement guidelines concerning inappropriate treatment of catheter-associated asymptomatic bacteriuria

<p>Abstract</p> <p>Background</p> <p>Catheter-associated urinary tract infection (CAUTI) is one of the most common hospital-acquired infections. However, many cases treated as hospital-acquired CAUTI are actually asymptomatic bacteriuria (ABU). Evidence-based guidelines recommend that providers neither screen for nor treat ABU in most catheterized patients, but there is a significant gap between these guidelines and clinical practice. Our objectives are (1) to evaluate the effectiveness of an audit and feedback intervention for increasing guideline-concordant care concerning catheter-associated ABU and (2) to measure improvements in healthcare providers' knowledge of and attitudes toward the practice guidelines associated with the intervention.</p> <p>Methods/Design</p> <p>The study uses a controlled pre/post design to test an intervention using audit and feedback of healthcare providers to improve their compliance with ABU guidelines. The intervention and the control sites are two VA hospitals. For objective 1 we will review medical records to measure the clinical outcomes of inappropriate screening for and treatment of catheter-associated ABU. For objective 2 we will survey providers' knowledge and attitudes. Three phases of our protocol are proposed: the first 12-month phase will involve observation of the baseline incidence of inappropriate screening for and treatment of ABU at both sites. This surveillance for clinical outcomes will continue at both sites throughout the study. Phase 2 consists of 12 months of individualized audit and feedback at the intervention site and guidelines distribution at both sites. The third phase, also over 12 months, will provide unit-level feedback at the intervention site to assess sustainability. Healthcare providers at the intervention site during phase 2 and at both sites during phase 3 will complete pre/post surveys of awareness and familiarity (knowledge), as well as of acceptance and outcome expectancy (attitudes) regarding the relevant practice guidelines.</p> <p>Discussion</p> <p>Our proposal to bring clinical practice in line with published guidelines has significant potential to decrease overdiagnosis of CAUTI and associated inappropriate antibiotic use. Our study will also provide information about how to maximize effectiveness of audit and feedback to achieve guideline adherence in the inpatient setting.</p> <p>Trial Registration</p> <p><a href="http://www.clinicaltrials.gov/ct2/show/NCT01052545">NCT01052545</a></p