Application of theory to enhance audit and feedback interventions to increase the uptake of evidence-based transfusion practice: an intervention development protocol

Background: Audits of blood transfusion demonstrate around 20% transfusions are outside national recommendations and guidelines. Audit and feedback is a widely used quality improvement intervention but effects on clinical practice are variable, suggesting potential for enhancement. Behavioural theory, theoretical frameworks of behaviour change and behaviour change techniques provide systematic processes to enhance intervention. This study is part of a larger programme of work to promote the uptake of evidence-based transfusion practice. Objectives: The objectives of this study are to design two theoretically enhanced audit and feedback interventions; one focused on content and one on delivery, and investigate the feasibility and acceptability. Methods: Study A (Content): A coding framework based on current evidence regarding audit and feedback, and behaviour change theory and frameworks will be developed and applied as part of a structured content analysis to specify the key components of existing feedback documents. Prototype feedback documents with enhanced content and also a protocol, describing principles for enhancing feedback content, will be developed. Study B (Delivery): Individual semi-structured interviews with healthcare professionals and observations of team meetings in four hospitals will be used to specify, and identify views about, current audit and feedback practice. Interviews will be based on a topic guide developed using the Theoretical Domains Framework and the Consolidated Framework for Implementation Research. Analysis of transcripts based on these frameworks will form the evidence base for developing a protocol describing an enhanced intervention that focuses on feedback delivery. Study C (Feasibility and Acceptability): Enhanced interventions will be piloted in four hospitals. Semi-structured interviews, questionnaires and observations will be used to assess feasibility and acceptability. Discussion: This intervention development work reflects the UK Medical Research Council’s guidance on development of complex interventions, which emphasises the importance of a robust theoretical basis for intervention design and recommends systematic assessment of feasibility and acceptability prior to taking interventions to evaluation in a full-scale randomised study. The work-up includes specification of current practice so that, in the trials to be conducted later in this programme, there will be a clear distinction between the control (usual practice) conditions and the interventions to be evaluated

Establishing a primary care audit and feedback implementation laboratory: a consensus study

Background: There is a significant variation among individual primary care providers in prescribing of potentially problematic, low-value medicines which cause avoidable patient harm. Audit and feedback is generally effective at improving prescribing. However, progress has been hindered by research waste, leading to unanswered questions about how to include audit and feedback for specific problems and circumstances. Trials of different ways of providing audit and feedback in implementation laboratories have been proposed as a way of improving population healthcare while generating robust evidence on feedback effects. However, there is limited experience in their design and delivery. Aim: To explore priorities, feasibility, and ethical challenges of establishing a primary care prescribing audit and feedback implementation laboratory. Design and setting: Two-stage Delphi consensus process involving primary care pharmacy leads, audit and feedback researchers, and patient and public. Method: Participants initially scored statements relating to priorities, feasibility, and ethical considerations for an implementation laboratory. These covered current feedback practice, priority topics for feedback, usefulness of feedback in improving prescribing and different types of prescribing data, acceptability and desirability of different organization levels of randomization, options for trial consent, different methods of delivering feedback, and interest in finding out how effective different ways of presenting feedback would be. After receiving collated results, participants then scored the items again. The consensus was defined using the GRADE criteria. The results were analyzed by group and overall score. Results: Fourteen participants reached consensus for 38 out of 55 statements. Addressing antibiotic and opioid prescribing emerged as the highest priorities for action. The panel supported statements around addressing highpriority prescribing issues, taking an “opt-out” approach to practice consent if waiving consent was not permitted, and randomizing at lower rather than higher organizational levels. Participants supported patient-level prescribing data and further research evaluating most of the different feedback methods we presented them with. Conclusions: There is a good level of support for evaluating a wide range of potential enhancements to improve the effects of feedback on prescribing. The successful design and delivery of a primary care audit and feedback implementation laboratory depend on identifying shared priorities and addressing practical and ethical considerations

A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants

Background: This paper presents the first meta-analysis for the inter-rater reliability (IRR) of journal peer reviews. IRR is defined as the extent to which two or more independent reviews of the same scientific document agree. Methodology/Principal Findings: Altogether, 70 reliability coefficients (Cohen’s Kappa, intra-class correlation [ICC], and Pearson product-moment correlation [r]) from 48 studies were taken into account in the meta-analysis. The studies were based on a total of 19,443 manuscripts; on average, each study had a sample size of 311 manuscripts (minimum: 28, maximum: 1983). The results of the meta-analysis confirmed the findings of the narrative literature reviews published to date: The level of IRR (mean ICC/r 2 =.34, mean Cohen’s Kappa =.17) was low. To explain the study-to-study variation of the IRR coefficients, meta-regression analyses were calculated using seven covariates. Two covariates that emerged in the metaregression analyses as statistically significant to gain an approximate homogeneity of the intra-class correlations indicated that, firstly, the more manuscripts that a study is based on, the smaller the reported IRR coefficients are. Secondly, if the information of the rating system for reviewers was reported in a study, then this was associated with a smaller IRR coefficient than if the information was not conveyed. Conclusions/Significance: Studies that report a high level of IRR are to be considered less credible than those with a low level o