Qualitative methods: An alternative view

Are qualitative and quantitative research really based on different philosophies? John Paley and Richard Lilford argue that the idea that qualitative research is constructivist should not be allowed to diffuse into medical literature unopposed

Stepped-wedge cluster randomised controlled trials : a generic framework including parallel and multiple-level designs

Stepped-wedge cluster randomised trials (SW-CRTs) are being used with increasing frequency in health service evaluation. Conventionally, these studies are cross-sectional in design with equally spaced steps, with an equal number of clusters randomised at each step and data collected at each and every step. Here we introduce several variations on this design and consider implications for power. One modification we consider is the incomplete cross-sectional SW-CRT, where the number of clusters varies at each step or where at some steps, for example, implementation or transition periods, data are not collected. We show that the parallel CRT with staggered but balanced randomisation can be considered a special case of the incomplete SW-CRT. As too can the parallel CRT with baseline measures. And we extend these designs to allow for multiple layers of clustering, for example, wards within a hospital. Building on results for complete designs, power and detectable difference are derived using a Wald test and obtaining the variance–covariance matrix of the treatment effect assuming a generalised linear mixed model. These variations are illustrated by several real examples. We recommend that whilst the impact of transition periods on power is likely to be small, where they are a feature of the design they should be incorporated. We also show examples in which the power of a SW-CRT increases as the intra-cluster correlation (ICC) increases and demonstrate that the impact of the ICC is likely to be smaller in a SW-CRT compared with a parallel CRT, especially where there are multiple levels of clustering. Finally, through this unified framework, the efficiency of the SW-CRT and the parallel CRT can be compared

Assessment of publication bias and outcome reporting bias in systematic reviews of health services and delivery research:A meta-epidemiological study

Strategies to identify and mitigate publication bias and outcome reporting bias are frequently adopted in systematic reviews of clinical interventions but it is not clear how often these are applied in systematic reviews relating to quantitative health services and delivery research (HSDR). We examined whether these biases are mentioned and/or otherwise assessed in HSDR systematic reviews, and evaluated associating factors to inform future practice. We randomly selected 200 quantitative HSDR systematic reviews published in the English language from 2007-2017 from the Health Systems Evidence database (www.healthsystemsevidence.org). We extracted data on factors that may influence whether or not authors mention and/or assess publication bias or outcome reporting bias. We found that 43% (n = 85) of the reviews mentioned publication bias and 10% (n = 19) formally assessed it. Outcome reporting bias was mentioned and assessed in 17% (n = 34) of all the systematic reviews. Insufficient number of studies, heterogeneity and lack of pre-registered protocols were the most commonly reported impediments to assessing the biases. In multivariable logistic regression models, both mentioning and formal assessment of publication bias were associated with: inclusion of a meta-analysis; being a review of intervention rather than association studies; higher journal impact factor, and; reporting the use of systematic review guidelines. Assessment of outcome reporting bias was associated with: being an intervention review; authors reporting the use of Grading of Recommendations, Assessment, Development and Evaluations (GRADE), and; inclusion of only controlled trials. Publication bias and outcome reporting bias are infrequently assessed in HSDR systematic reviews. This may reflect the inherent heterogeneity of HSDR evidence and different methodological approaches to synthesising the evidence, lack of awareness of such biases, limits of current tools and lack of pre-registered study protocols for assessing such biases. Strategies to help raise awareness of the biases, and methods to minimise their occurrence and mitigate their impacts on HSDR systematic reviews, are needed

Stakeholder views on publication bias in health services research

Objectives: While the presence of publication bias in clinical research is well documented, little is known about its role in the reporting of health services research. This paper explores stakeholder perceptions and experiences with regard to the role of publication and related biases in quantitative research relating to the quality, accessibility and organization of health services. Methods: We present findings from semi-structured interviews with those responsible for the funding, publishing and/or conduct of quantitative health services research, primarily in the UK. Additional data collection includes interviews with health care decision makers as ‘end users’ of health services research, and a focus group with patient and service user representatives. The final sample comprised 24 interviews and eight focus group participants. Results: Many study participants felt unable to say with any degree of certainty whether publication bias represents a significant problem in quantitative health services research. Participants drew broad contrasts between externally funded and peer reviewed research on the one hand, and end user funded quality improvement projects on the other, with the latter perceived as more vulnerable to selective publication and author over-claiming. Multiple study objectives, and a general acceptance of ‘mess and noise’ in the data and its interpretation was seen to reduce the importance attached to replicable estimates of effect sizes in health services research. The relative absence of external scrutiny, either from manufacturers of interventions or health system decision makers, added to this general sense of ‘low stakes’ of health services research. As a result, while many participants advocated study pre-registration and using protocols to pre-identify outcomes, others saw this as an unwarranted imposition. Conclusions: This study finds that incentives towards publication and related bias are likely to be present, but not to the same degree as in clinical research. In health services research, these were seen as being offset by other forms of ‘novelty’ bias in the reporting and publishing of research findings

Adaption, implementation and evaluation of collaborative service improvements in the testing and result communication process in primary care from patient and staff perspectives : a qualitative study

BACKGROUND: Increasing numbers of blood tests are being ordered in primary care settings and the swift and accurate communication of test results is central to providing high quality care. The process of testing and result communication is complex and reliant on the coordinated actions of care providers, external groups in laboratory and hospital settings, and patients. This fragmentation leaves it vulnerable to error and the need to improve an apparently fallible system is apparent. However, primary care is complex and does not necessarily adopt change in a linear and prescribed manner influenced by a range of factors relating to practice staff, patients and organisational factors. To account for these competing perspectives, we worked in conjunction with both staff and patients to develop and implement strategies intended to improve patient satisfaction and increase efficiency of existing processes. METHODS: The study applied the principles of 'experience-based co-design' to identify key areas of weakness and source proposals for change from staff and patients. The study was undertaken within two primary practices situated in South Birmingham (UK) of contrasting size and socio-economic environment. Senior practice staff were involved in the refinement of the interventions for introduction. We conducted focus groups singly constituted of staff and patients at each practice to determine suitability, applicability and desirability alongside the practical implications of their introduction. RESULTS: At each practice four of the six proposals for change were implemented these were increased access to phlebotomy, improved receptionist training, proactive communication of results, and increased patient awareness of the tests ordered and the means of their communication. All were received favourably by both patients and staff. The remaining issues around the management of telephone calls and the introduction of electronic alerts for missing results were not addressed due to constraints of time and available resources. CONCLUSIONS: Approaches to tackling the same area of weakness differed at practices and was determined by individual staff attitudes and by organisational and patient characteristics. The long-term impact of the changes requires further quantitative evaluation

A library of logic models to explain how interventions to reduce diagnostic error work

OBJECTIVES: We aimed to create a library of logic models for interventions to reduce diagnostic error. This library can be used by those developing, implementing, or evaluating an intervention to improve patient care, to understand what needs to happen, and in what order, if the intervention is to be effective. METHODS: To create the library, we modified an existing method for generating logic models. The following five ordered activities to include in each model were defined: preintervention; implementation of the intervention; postimplementation, but before the immediate outcome can occur; the immediate outcome (usually behavior change); and postimmediate outcome, but before a reduction in diagnostic errors can occur. We also included reasons for lack of progress through the model. Relevant information was extracted about existing evaluations of interventions to reduce diagnostic error, identified by updating a previous systematic review. RESULTS: Data were synthesized to create logic models for four types of intervention, addressing five causes of diagnostic error in seven stages in the diagnostic pathway. In total, 46 interventions from 43 studies were included and 24 different logic models were generated. CONCLUSIONS: We used a novel approach to create a freely available library of logic models. The models highlight the importance of attending to what needs to occur before and after intervention delivery if the intervention is to be effective. Our work provides a useful starting point for intervention developers, helps evaluators identify intermediate outcomes, and provides a method to enable others to generate libraries for interventions targeting other errors

Peri-operative pulse oximetry in low-income countries: a cost–effectiveness analysis

Abstract Objective: To evaluate the cost–effectiveness of pulse oximetry – compared with no peri-operative monitoring – during surgery in low-income countries. Methods: We considered the use of tabletop and portable, hand-held pulse oximeters among patients of any age undergoing major surgery in low-income countries. From earlier studies we obtained baseline mortality and the effectiveness of pulse oximeters to reduce mortality. We considered the direct costs of purchasing and maintaining pulse oximeters as well as the cost of supplementary oxygen used to treat hypoxic episodes identified by oximetry. Health benefits were measured in disability-adjusted life-years (DALYs) averted and benefits and costs were both discounted at 3% per year. We used recommended cost–effectiveness thresholds – both absolute and relative to gross domestic product (GDP) per capita – to assess if pulse oximetry is a cost–effective health intervention. To test the robustness of our results we performed sensitivity analyses. Findings: In 2013 prices, tabletop and hand-held oximeters were found to have annual costs of 310 and 95 United States dollars (US$), respectively. Assuming the two types of oximeter have identical effectiveness, a single oximeter used for 22 procedures per week averted 0.83 DALYs per annum. The tabletop and hand-held oximeters cost US$ 374 and US$115 per DALY averted, respectively. For any country with a GDP per capita above US$ 677 the hand-held oximeter was found to be cost–effective if it prevented just 1.7% of anaesthetic-related deaths or 0.3% of peri-operative mortality. Conclusion: Pulse oximetry is a cost–effective intervention for low-income settings

Bayesian Cohort and Cross-Sectional Analyses of the PINCER Trial: A Pharmacist-Led Intervention to Reduce Medication Errors in Primary Care

Background Medication errors are an important source of potentially preventable morbidity and mortality. The PINCER study, a cluster randomised controlled trial, is one of the world’s first experimental studies aiming to reduce the risk of such medication related potential for harm in general practice. Bayesian analyses can improve the clinical interpretability of trial findings. Methods Experts were asked to complete a questionnaire to elicit opinions of the likely effectiveness of the intervention for the key outcomes of interest - three important primary care medication errors. These were averaged to generate collective prior distributions, which were then combined with trial data to generate Bayesian posterior distributions. The trial data were analysed in two ways: firstly replicating the trial reported cohort analysis acknowledging pairing of observations, but excluding non-paired observations; and secondly as cross-sectional data, with no exclusions, but without acknowledgement of the pairing. Frequentist and Bayesian analyses were compared. Findings Bayesian evaluations suggest that the intervention is able to reduce the likelihood of one of the medication errors by about 50 (estimated to be between 20% and 70%). However, for the other two main outcomes considered, the evidence that the intervention is able to reduce the likelihood of prescription errors is less conclusive. Conclusions Clinicians are interested in what trial results mean to them, as opposed to what trial results suggest for future experiments. This analysis suggests that the PINCER intervention is strongly effective in reducing the likelihood of one of the important errors; not necessarily effective in reducing the other errors. Depending on the clinical importance of the respective errors, careful consideration should be given before implementation, and refinement targeted at the other errors may be something to consider