The challenges faced in the design, conduct and analysis of surgical randomised trials

Randomised evaluations of surgical interventions are rare; some interventions have been widely adopted without rigorous evaluation. Unlike other medical areas, the randomised controlled trial (RCT) design has not become the default study design for the evaluation of surgical interventions. Surgical trials are difficult to successfully undertake and pose particular practical and methodological challenges. However, RCTs have played a role in the assessment of surgical innovations and there is scope and need for greater use. This article will consider the design, conduct and analysis of an RCT of a surgical intervention. The issues will be reviewed under three headings: the timing of the evaluation, defining the research question and trial design issues. Recommendations on the conduct of future surgical RCTs are made. Collaboration between research and surgical communities is needed to address the distinct issues raised by the assessment of surgical interventions and enable the conduct of appropriate and well-designed trials

Using the literature to quantify the learning curve: a case study

Objective: To assess whether a literature review of a technology can allow a learning curve to be quantified. Methods: The literature for fibreoptic intubation was searched for studies reporting information relevant to the learning curve. The Cochrane Librar y, Medline, Embase and Science Citation index were searched. Studies that reported a procedure time were included. Data were abstracted on the three features of learning: initial level, rate of learning and asymptote level. Random effect meta-analysis was performed. Results: Only 21 studies gave explicit information concerning the previous experience of the operator(s). There were 32 different definitions of procedure time. From 4 studies of fibreoptic nasotracheal intubation, the mean starting level and time for the 10th procedure (95% confidence interval) was estimated to be 133s (113, 153) and 71s (62, 79) respectively. Conclusions: The review approach allowed learning to be quantified for our example technology. Poor and insufficient reporting constrained formal statistical estimation. Standardised reporting of non-drug techniques with adequate learning curve details is needed to inform trial design and costeffectiveness analysis

The curious case of an internal pilot in a multicentre randomised trial-time for a rethink?

Acknowledgements The authors would like to acknowledge helpful discussions on this topic held with various colleagues in connection with a variety of projects. Funding No funding was received for this work.Peer reviewedPublisher PD

Randomised controlled trial comparing rubber band ligation with stapled haemorrhoidopexy for Grade II circumferential haemorrhoids : Long-term results

Peer reviewedPostprin

Outcomes of stenting after uncomplicated ureteroscopy: systematic review and meta-analysis.

Objective To investigate the potential beneficial and adverse effects of routine ureteric stent placement after ureteroscopy. Design Systematic review and meta-analysis of randomised controlled trials. Data sources Cochrane controlled trials register (2006 issue 2), Embase, and Medline (1966 to 31 March 2006), without language restrictions. Review methods We included all randomised controlled trials that reported various outcomes with or without stenting after ureteroscopy. Two reviewers independently extracted data and assessed quality. Meta-analyses used both fixed and random effects models with dichotomous data reported as relative risk and continuous data as a weighted mean difference with 95% confidence intervals. Results Nine randomised controlled trials (reporting 831 participants) were identified. The incidence of lower urinary tract symptoms was significantly higher in participants who had a stent inserted (relative risk 2.25, 95% confidence interval 1.14 to 4.43, for dysuria; 2.00, 1.11 to 3.62, for frequency or urgency) after ureteroscopy. There was no significant difference in postoperative requirement for analgesia, urinary tract infections, stone free rate, and ureteric strictures in the two groups. Because of marked heterogeneity, formal pooling of data was not possible for some outcomes such as flank pain. A pooled analysis showed a reduced likelihood of unplanned medical visits or admission to hospital in the group with stents (0.53, 0.17 to 1.60), although this difference was not significant. None of the trials reported on health related quality of life. Cost reported in three randomised controlled trials favoured the group without stents. The overall quality of trials was poor and reporting of outcomes inconsistent. Conclusions Patients with stents after ureteroscopy have significantly higher morbidity in the form of irritative lower urinary symptoms with no influence on stone free rate, rate of urinary tract infection, requirement for analgesia, or long term ureteric stricture formation. Because of the marked heterogeneity and poor quality of reporting of the included trials, the place of stenting in the managementof patients after uncomplicated ureteroscopy remains unclear.Peer reviewedPublisher PD

64-slice computed tomography angiography in the diagnosis and assessment of coronary artery disease : systematic review and meta-analysis

Objective To assess whether 64-slice computed tomography (CT) angiography might replace some coronary angiography (CA) for diagnosis and assessment of coronary artery disease (CAD). Methods We searched electronic databases, conference proceedings and scanned reference lists of included studies. Eligible studies compared 64-slice CT with a reference standard of CA in adults with suspected/known CAD, reporting sensitivity and specificity or true and false positives and negatives. Data were pooled using the hierarchical summary receiver operating characteristic model. Results Forty studies were included; 28 provided sufficient data for inclusion in the meta-analyses, all using a cutoff of ≥ 50% stenosis to define significant CAD. In patient-based detection (n=1286) 64-slice CT pooled sensitivity was 99% (95% credible interval (CrI) 97 to 99%), specificity 89% (95% CrI 83 to 94%), median positive predictive value (PPV) across studies 93% (range 64 to 100%) and negative predictive value (NPV) 100% (range 86 to 100%). In segment-based detection (n=14,199) 64-slice CT pooled sensitivity was 90% (95% CrI 85 to 94%), specificity 97% (95% CrI 95 to 98%), median positive predictive value (PPV) across studies 76% (range 44 to 93%) and negative predictive value (NPV) 99% (range 95 to 100%). Conclusions 64-slice CT is highly sensitive for patient-based detection of CAD and has high NPV. An ability to rule out significant CAD means that it may have a role in the assessment of chest pain, particularly when the diagnosis remains uncertain despite clinical evaluation and simple non-invasive testing.UK National Institute for Health Research Health Technology Assessment programme (project number 06/15/01). The Health Services Research Unit is core funded by the Chief Scientist Office of the Scottish Government Health Directorates.Peer reviewedAuthor versio

Challenges in evaluating surgical innovation

Research on surgical interventions is associated with several methodological and practical challenges of which few, if any, apply only to surgery. However, surgical evaluation is especially demanding because many of these challenges coincide. In this report, the second of three on surgical innovation and evaluation, we discuss obstacles related to the study design of randomised controlled trials and non-randomised studies assessing surgical interventions. We also describe the issues related to the nature of surgical procedures—for example, their complexity, surgeon-related factors, and the range of outcomes. Although difficult, surgical evaluation is achievable and necessary. Solutions tailored to surgical research and a framework for generating evidence on which to base surgical practice are essential.The Balliol Colloquium has been supported by Ethicon UK with unrestricted educational grants and by the National Institute of Health Research Health Technology Assessment Programme. The Balliol Colloquium was administratively and financially supported by the Nuffield Department of Surgery at the University of Oxford and the Department of Surgery at McGill University. JAC holds a Medical Research Council UK special training fellowship. The University of Aberdeen’s Health Services Research Unit is core funded by the Chief Scientist Offi ce of the Scottish Government Health Directorates. IB is supported by a grant from the Société Française de Rhumatologie and Lavoisier Program (Ministère des Aff aires Etrangères et Européennes). PLE is a DPhil Candidate in Evidence-Based Health Care at Oxford University

A pragmatic multi-centre randomised controlled trial of fluid loading and level of dependency in high-risk surgical patients undergoing major elective surgery : trial protocol

Peer reviewedPublisher PD

Systematic review of the clinical effectiveness and cost-effectiveness of 64-slice or higher computed tomography angiography as an alternative to invasive coronary angiography in the investigation of coronary artery disease

Objectives: To assess the clinical effectiveness and cost-effectiveness, in different patient groups, of the use of 64-slice or higher computed tomography (CT) angiography, instead of invasive coronary angiography (CA), for diagnosing people with suspected coronary artery disease (CAD) and assessing people with known CAD. Data sources: Electronic databases were searched from 2002 to December 2006. Review methods: Included studies were tabulated and sensitivity, specificity, positive and negative predictive values calculated. Meta-analysis models were fitted using hierarchical summary receiver operating characteristic curves. Summary sensitivity, specificity, positive and negative likelihood ratios and diagnostic odds ratios for each model were reported as a median and 95% credible interval (CrI). Searches were also carried out for studies on the cost-effectiveness of 64-slice CT in the assessment of CAD. Results: The diagnostic accuracy and prognostic studies enrolled over 2500 and 1700 people, respectively. The overall quality of the studies was reasonably good. In the pooled estimates, 64-slice CT angiography was highly sensitive (99%, 95% CrI 97 to 99%) for patientbased detection of significant CAD (defined as 50% or more stenosis), while across studies the negative predictive value (NPV) was very high (median 100%, range 86 to 100%). In segment-level analysis compared with patient-based detection, sensitivity was lower (90%, 95% CrI 85 to 94%, versus 99%, 95% CrI 97 to 99%) and specificity higher (97%, 95% CrI 95 to 98%, versus 89%, 95% CrI 83 to 94%), while across studies the median NPV was similar (99%, range 95 to 100%, versus 100%, range 86 to 100%). At individual coronary artery level the pooled estimates for sensitivity ranged from 85% for the left circumflex (LCX) artery to 95% for the left main artery, specificity ranged from 96% for both the left anterior descending (LAD) artery and LCX to 100% for the left main artery, while across studies the positive predictive value (PPV) ranged from 81% for the LCX to 100% for the left main artery and NPV was very high, ranging from 98% for the LAD (range 95 to 100%), LCX (range 93 to 100%) and right coronary artery (RCA) (range 94 to 100%) to 100% for the left main artery. The pooled estimates for bypass graft analysis were 99% (95% CrI 95 to 100%) sensitivity, 96% (95% CrI 86 to 99%) specificity, with median PPV and NPV values across studies of 93% (range 90 to 95%) and 99% (range 98 to 100%), respectively. This compares with, for stent analysis, a pooled sensitivity of 89% (95% CrI 68 to 97%), specificity 94% (95% CrI 83 to 98%), and median PPV and NPV values across studies of 77% (range 33 to 100%) and 96% (range 71 to 100%), respectively. Sixty-four-slice CT is almost as good as invasive CA in terms of detecting true positives. However, it is somewhat poorer in its rate of false positives. It seems likely that diagnostic strategies involving 64-slice CT will still require invasive CA for CT test positives, partly to identify CT false positives, but also because CA provides other information that CT currently does not, notably details of insertion site and distal run-off for possible coronary artery bypass graft (CABG). The high sensitivity of 64-slice CT avoids the costs of unnecessary CA in those referred for investigation but who do not have CAD. Given the possible, although small, associated death rate, avoiding these unnecessary CAs through the use of 64-slice CT may also confer a small immediate survival advantage. This in itself may be sufficient to outweigh the very marginally inferior rates of detection of true positives by strategies involving 64-slice CT. The avoidance of unnecessary CA through the use of 64-slice CT also appears likely to result in overall cost savings in the diagnostic pathway. Only if both the cost of CA is relatively low and the prevalence of CAD in the presenting population is relatively high (so that most patients will go on to CA) will the use of 64-slice CT be likely to result in a higher overall diagnostic cost per patient. Conclusions: The main value of 64-slice CT may at present be to rule out significant CAD. It is unlikely to replace CA in assessment for revascularisation of patients, particularly as angiography and angioplasty are often done on the same occasion. Further research is needed into the marginal advantages and costs of 256-slice machines compared with 64-sliceCT, the usefulness of 64-slice CT in people with suspected acute coronary syndrome, the potential of multislice computed tomography to examine plaque morphology, the role of CT in identifying patients suitable for CABG, and the concerns raised about repetitive use, or use of 64-slice or higher CT angiography in younger individuals or women of childbearing age.The Health Services Research Unit, Institute of Applied Health Sciences, University of Aberdeen, is core-funded by the Chief Scientist Office of the Scottish Government Health Directorates

A cost-utility analysis of multivitamin and multimineral supplements in men and women aged 65 years and over

Background and Aims As people age there is a progressive dysregulation of the immune system that may lead to an increased risk of infections, which may precipitate hospital admission in people with chronic heart or respiratory diseases. Mineral and vitamin supplementation in older people could therefore influence infections in older people. However, the evidence from the available randomised controlled trials is mixed. The aim of the study was to assess the relative efficiency of multivitamin and multimineral supplementation compared with no supplementation. Methods Cost-utility analysis alongside a randomised controlled trial. Participants aged 65 years or over from six general practices in Grampian, Scotland were studied. They were randomised to one tablet daily of either a multivitamin and multimineral supplement or matching placebo. Exclusion criteria were: use of mineral, vitamin or fish oil supplements in the previous three months (one month for water soluble vitamins), vitamin B12 injection in the last three months. Results Nine hundred and ten participants were recruited (454 placebo and 456 supplementation). Use of health service resources and costs were similar between the two groups. The supplementation arm was more costly although this was not statistically significant (£15 per person, 95% CI -3.75 to 34.95). After adjusting for minimisation and baseline EQ-5D scores supplementation was associated with fewer QALYs per person (–0.018, 95% CI -0.04 to 0.002). It was highly unlikely that supplementation would be considered cost-effective. Conclusions The evidence from this study suggests that it is highly unlikely that supplementation could be considered cost-effective.Peer reviewedAuthor versio