Monitoring the performance of residents during training in off-pump coronary surgery.

OBJECTIVE: Control charts (eg, cumulative sum charts) plot changes in performance with time and can alert a surgeon to suboptimal performance. They were used to compare performance of off-pump coronary artery bypass surgery between a consultant and four resident surgeons and to compare performance of off-pump coronary artery bypass surgery and conventional coronary artery bypass grafting within surgeons. METHODS: Data were analyzed for consecutive patients undergoing coronary artery bypass grafting who were operated on by one consultant or one of four residents. Conversions were analyzed by intention to treat. Perioperative death or one or more of 10 adverse events constituted failure. Predicted risks of failure for individual patients were derived from the study population. Variable life-adjusted displays and risk-adjusted sequential probability ratio test charts were plotted. RESULTS: Data for 1372 patients were analyzed; 769 of the procedures were off-pump coronary artery bypass operations (56.0%). The consultant operated on 382 patients (293 off-pump, 76.7%), and the residents operated on 990 (474 off-pump, 47.9%). Patients operated on by residents tended to be older, more obese, more likely to require an urgent operation, and more likely to need a circumflex artery graft but less likely to have triple-vessel disease. There were 7 conversions (consultant 5, residents 2). The overall failure rate was 8.5% (9.2% for consultant's operations and 8.2% for residents' operations), including 10 deaths (0.7%). Predicted and observed risks of failure were similar for all five surgeons. After 100 off-pump coronary artery bypass operations, performance was the same or better for the residents as for the consultant. For all surgeons, performance was the same or better for off-pump as for conventional coronary artery bypass grafting. CONCLUSIONS: Off-pump coronary artery bypass surgery can be safely taught to cardiothoracic residents. Implementation of continuous performance monitoring for residents is practicable

Approaches to multiplicity in publicly funded pragmatic randomised controlled trials:a survey of clinical trials units and a rapid review of published trials

BACKGROUND: Opinions and practices vary around the issue of performing multiple statistical tests in randomised controlled trials (RCTs). We carried out a study to collate information about opinions and practices using a methodological rapid review and a survey, specifically of publicly funded pragmatic RCTs that are not seeking marketing authorisation. The aim was to identify the circumstances under which researchers would make a statistical adjustment for multiplicity. METHODS: A review was performed extracting information from articles reporting primary analyses of pragmatic RCTs in one of seven high quality medical journals, in January to June (inclusive) 2018. A survey (Survey Monkey) eliciting opinions and practices around multiplicity was distributed to the 47 registered clinical trials units (CTUs) in the UK. RESULTS: One hundred and thirty-eight RCTs were included in the review, and survey responses were received from 27/47 (57%) CTUs. Both the review and survey indicated that adjusting for multiplicity was considered most important for multiple treatment comparisons; adjustment was performed for 11/23 (48%) published trials, and 24/27 (89%) CTU statisticians reported they would consider adjustment. Opinions and practices varied around adjustment for multiplicity arising from multiple primary outcomes and interim analyses. Adjustment was considered less important for multiplicity due to multiple secondary outcomes (adjustment performed for 17/136 [13%] published trials and 3/27 [11%] CTU statisticians would consider adjustment) and subgroup analyses (8/85 [9%] published trials adjusted and 6/27 CTU [22%] statisticians would consider adjustment). CONCLUSIONS: There is variation in opinions about adjustment for multiplicity among both statisticians reporting RCTs and applied statisticians working in CTUs. Further guidance is needed on the circumstances in which adjustment should be considered in relation to primary trial hypotheses, and if there are any situations in which adjustment would be recommended in the context of secondary analyses. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12874-022-01525-9

Applicability and generalisability of published results of randomised controlled trials and non-randomised studies evaluating four orthopaedic procedures: methodological systematic review

Objective To compare the reporting of essential applicability data from randomised controlled trials and non-randomised studies evaluating four new orthopaedic surgical procedures

Challenges to implementing the national programme for information technology (NPfIT): a qualitative study.

OBJECTIVES: To describe the context for implementing the national programme for information technology (NPfIT) in England, actual and perceived barriers, and opportunities to facilitate implementation. DESIGN: Case studies and in depth interviews, with themes identified using a framework developed from grounded theory. SETTING: Four acute NHS trusts in England. PARTICIPANTS: Senior trust managers and clinicians, including chief executives, directors of information technology, medical directors, and directors of nursing. RESULTS: The trusts varied in their circumstances, which may affect their ability to implement the NPfIT. The process of implementation has been suboptimal, leading to reports of low morale by the NHS staff responsible for implementation. The overall timetable is unrealistic, and trusts are uncertain about their implementation schedules. Short term benefits alone are unlikely to persuade NHS staff to adopt the national programme enthusiastically, and some may experience a loss of electronic functionality in the short term. CONCLUSIONS: The sociocultural challenges to implementing the NPfIT are as daunting as the technical and logistical ones. Senior NHS staff feel these have been neglected. We recommend that national programme managers prioritise strategies to improve communication with, and to gain the cooperation of, front line staff

Challenges in evaluating surgical innovation

Research on surgical interventions is associated with several methodological and practical challenges of which few, if any, apply only to surgery. However, surgical evaluation is especially demanding because many of these challenges coincide. In this report, the second of three on surgical innovation and evaluation, we discuss obstacles related to the study design of randomised controlled trials and non-randomised studies assessing surgical interventions. We also describe the issues related to the nature of surgical procedures—for example, their complexity, surgeon-related factors, and the range of outcomes. Although difficult, surgical evaluation is achievable and necessary. Solutions tailored to surgical research and a framework for generating evidence on which to base surgical practice are essential.The Balliol Colloquium has been supported by Ethicon UK with unrestricted educational grants and by the National Institute of Health Research Health Technology Assessment Programme. The Balliol Colloquium was administratively and financially supported by the Nuffield Department of Surgery at the University of Oxford and the Department of Surgery at McGill University. JAC holds a Medical Research Council UK special training fellowship. The University of Aberdeen’s Health Services Research Unit is core funded by the Chief Scientist Offi ce of the Scottish Government Health Directorates. IB is supported by a grant from the Société Française de Rhumatologie and Lavoisier Program (Ministère des Aff aires Etrangères et Européennes). PLE is a DPhil Candidate in Evidence-Based Health Care at Oxford University

Implementation of computerised physician order entry (CPOE) and picture archiving and communication systems (PACS) in the NHS: quantitative before and after study

Objective To assess the impact of components of the national programme for information technology (NPfIT) on measures of clinical and operational efficiency

Quasi-experimental study designs series-paper 5: a checklist for classifying studies evaluating the effects on health interventions-a taxonomy without labels.

OBJECTIVES: The aim of the study was to extend a previously published checklist of study design features to include study designs often used by health systems researchers and economists. Our intention is to help review authors in any field to set eligibility criteria for studies to include in a systematic review that relate directly to the intrinsic strength of the studies in inferring causality. We also seek to clarify key equivalences and differences in terminology used by different research communities. STUDY DESIGN AND SETTING: Expert consensus meeting. RESULTS: The checklist comprises seven questions, each with a list of response items, addressing: clustering of an intervention as an aspect of allocation or due to the intrinsic nature of the delivery of the intervention; for whom, and when, outcome data are available; how the intervention effect was estimated; the principle underlying control for confounding; how groups were formed; the features of a study carried out after it was designed; and the variables measured before intervention. CONCLUSION: The checklist clarifies the basis of credible quasi-experimental studies, reconciling different terminology used in different fields of investigation and facilitating communications across research communities. By applying the checklist, review authors' attention is also directed to the assumptions underpinning the methods for inferring causality