Evaluating interventions to make healthcare safer : methodological analysis and case study

This thesis describes study designs for the robust evaluation of complex patient safety interventions. Fundamentally, study designs available to measure the effectiveness of patient safety interventions fall into two categories – those that use contemporaneous controls, and those that do not. A review of the recent literature (245 citations) revealed that most studies were single-centre (63%), and the majority of these did not use contemporaneous controls (84%); whilst in multi-centre studies (37%) the number of studies using contemporaneous controls (49%) equalled the number of studies that that did not (51%). Studies that do not use contemporaneous controls dominate the literature, but they are weak and subject to bias. The thesis further discussed a case-study, as an exemplar for the evaluation of a highly complex patient safety intervention – the Safer Patients Initiative (SPI), which sought to generically strengthen hospitals, whilst improving frontline activities. The evaluation was a before and after study, with contemporaneous controls. It used mixed-methods, so that the triangulation of a one type of research finding could be reinforced when corroborated by the finding of another type. Uniquely, it also, compared the rates of change across control and SPI hospitals – an approach referred to as the “difference-in-difference” method.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Evaluating interventions to make healthcare safer: methodological analysis and case study

This thesis describes study designs for the robust evaluation of complex patient safety interventions. Fundamentally, study designs available to measure the effectiveness of patient safety interventions fall into two categories – those that use contemporaneous controls, and those that do not. A review of the recent literature (245 citations) revealed that most studies were single-centre (63%), and the majority of these did not use contemporaneous controls (84%); whilst in multi-centre studies (37%) the number of studies using contemporaneous controls (49%) equalled the number of studies that that did not (51%). Studies that do not use contemporaneous controls dominate the literature, but they are weak and subject to bias. The thesis further discussed a case-study, as an exemplar for the evaluation of a highly complex patient safety intervention – the Safer Patients Initiative (SPI), which sought to generically strengthen hospitals, whilst improving frontline activities. The evaluation was a before and after study, with contemporaneous controls. It used mixed-methods, so that the triangulation of a one type of research finding could be reinforced when corroborated by the finding of another type. Uniquely, it also, compared the rates of change across control and SPI hospitals – an approach referred to as the “difference-in-difference” method

Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase

ABSTRACT Objective To independently evaluate the impact of the second phase of the Health Foundation's Safer Patients Initiative (SPI2) on a range of patient safety measures. Design A controlled before and after design. Five substudies: survey of staff attitudes; review of case notes from high risk (respiratory) patients in medical wards; review of case notes from surgical patients; indirect evaluation of hand hygiene by measuring hospital use of handwashing materials; measurement of outcomes (adverse events, mortality among high risk patients admitted to medical wards, patients' satisfaction, mortality in intensive care, rates of hospital acquired infection). Setting NHS hospitals in England. Participants Nine hospitals participating in SPI2 and nine matched control hospitals. Intervention The SPI2 intervention was similar to the SPI1, with somewhat modified goals, a slightly longer intervention period, and a smaller budget per hospital. Results One of the scores (organisational climate) showed a significant (P=0.009) difference in rate of change over time, which favoured the control hospitals, though the difference was only 0.07 points on a five point scale. Results of the explicit case note reviews of high risk medical patients showed that certain practices improved over time in both control and SPI2 hospitals (and none deteriorated), but there were no significant differences between control and SPI2 hospitals. Monitoring of vital signs improved across control and SPI2 sites. This temporal effect was significant for monitoring the respiratory rate at both the six hour (adjusted odds ratio 2.1, 99% confidence interval 1.0 to 4.3; P=0.010) and 12 hour (2.4, 1.1 to 5.0; P=0.002) periods after admission. There was no significant effect of SPI for any of the measures of vital signs. Use of a recommended system for scoring the severity of pneumonia improved from 1.9% (1/52) to 21.4% (12/56) of control and from 2.0% (1/50) to 41.7% (25/60) of SPI2 patients. This temporal change was significant (7.3, 1.4 to 37.7; P=0.002), but the difference in difference was not significant (2.1, 0.4 to 11.1; P=0.236). There were no notable or significant changes in the pattern of prescribing errors, either over time or between control and SPI2 hospitals. Two items of medical history taking (exercise tolerance and occupation) showed significant improvement over time, across both control and SPI2 hospitals, but no additional SPI2 effect. The holistic review showed no significant changes in error rates either over time or between control and SPI2 hospitals. The explicit case note review of perioperative care showed that adherence rates for two of the four perioperative standards targeted by SPI2 were already good at baseline, exceeding 94% for antibiotic prophylaxis and 98% for deep vein thrombosis prophylaxis. Intraoperative monitoring of temperature improved over time in both groups, but this was not significant (1.8, 0.4 to 7.6; P=0.279), and there were no additional effects of SPI2. A dramatic rise in consumption of soap and alcohol hand rub was similar in control and SPI2 hospitals (P=0.760 and P=0.889, respectively), as was the corresponding decrease in rates of Clostridium difficile and meticillin resistant Staphylococcus aureus infection (P=0.652 and P=0.693, respectively). Mortality rates of medical patients included in the case note reviews in control hospitals increased from 17.3% (42/ 243) to 21.4% (24/112), while in SPI2 hospitals they fell from 10.3% (24/233) to 6.1% (7/114) (P=0.043). Fewer than 8% of deaths were classed as avoidable; changes in proportions could not explain the divergence of overall death rates between control and SPI2 hospitals. There was no significant difference in the rate of change in mortality in intensive care. Patients' satisfaction improved in both control and SPI2 hospitals on all dimensions, but again there were no significant changes between the two groups of hospitals

An epistemology of patient safety research: a framework for study design and interpretation. Article 2 Study Design

This is the second in a four-part series of articles detailing the epistemology of patient safety research. This article concentrates on issues of study design. It first considers the range of designs that may be used in the evaluation of patient safety interventions, highlighting the circumstances in which each is appropriate. The paper then provides details about an innovative study design, the stepped wedge, which may be particularly appropriate in the context of patient safety interventions, since these are expected to do more good than harm. The unit of allocation in patient safety research is also considered, since many interventions need to be delivered at cluster or service level. The paper also discusses the need to ensure the masking of patients, caregivers, observers and analysts wherever possible to minimise information biases and the Hawthorne effect. The difficulties associated with masking in patient safety research are described and suggestions given on how these can be ameliorated. The paper finally considers the role of study design in increasing confidence in the generalisability of study results over time and place. The extent to which findings can be generalised over time and place should be considered as part of an evaluation, for example by undertaking qualitative or quantitative measures of fidelity, attitudes or subgroup effects