Skip to main content
Article thumbnail
Location of Repository

Challenges in evaluating surgical innovation

By Patrick L Ergina, Jane M Blazeby, Pierre-Alain Clavein, Barnaby C Reeves, Christoph M Seiler, Isabelle Boutron and Jonathan Alistair Cook


Research on surgical interventions is associated with several methodological and practical challenges of which few, if any, apply only to surgery. However, surgical evaluation is especially demanding because many of these challenges coincide. In this report, the second of three on surgical innovation and evaluation, we discuss obstacles related to the study design of randomised controlled trials and non-randomised studies assessing surgical interventions. We also describe the issues related to the nature of surgical procedures—for example, their complexity, surgeon-related factors, and the range of outcomes. Although difficult, surgical evaluation is achievable and necessary. Solutions tailored to surgical research and a framework for generating evidence on which to base surgical practice are essential.The Balliol Colloquium has been supported by Ethicon UK with unrestricted educational grants and by the National Institute of Health\ud Research Health Technology Assessment Programme. The Balliol Colloquium was administratively and financially supported by the Nuffield Department of Surgery at the University of Oxford and the Department of Surgery at McGill University. JAC holds a Medical Research Council UK special training fellowship. The University of Aberdeen’s Health Services Research Unit is core funded by the Chief Scientist Offi ce of the Scottish Government Health Directorates. IB is supported by a grant from the Société Française de Rhumatologie and Lavoisier Program (Ministère des Aff aires Etrangères et Européennes). PLE is a DPhil Candidate in Evidence-Based Health Care at Oxford\ud University

Topics: Biomedical research, Surgical procedures, operative
Publisher: Elsevier
Year: 2009
OAI identifier:

Suggested articles


  1. (2006). A blended knowledge translation initiative to improve colorectal cancer staging [ISRCTN56824239]. BMC Health Serv Res.
  2. A concept for trial institutions focussing on randomised controlled trials in surgery. doi
  3. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. doi
  4. (2000). A fair way of donating hearts for transplantation. BMJ doi
  5. (2009). An evidence-based medicine review of lymphadenectomy extent for gastric cancer.
  6. Any casualties in the clash of randomized and observational evidence?
  7. (2008). ASERNIP-S: International trend setting. doi
  8. (2006). Assessment of Complications After Pancreatic Surgery: A Novel Grading System Applied to 633 Patients Undergoing Pancreaticoduodenectomy. Ann Surg doi
  9. (2006). CABG is still the best treatment for multivessel and left main disease, but patients need to know. Ann Thorac Surg doi
  10. (2008). Chapter 13. Including nonrandomized studies.
  11. (2008). Chapter 8: Assessing the risk of bias in included studies. doi
  12. Choosing between randomized and non-randomised studies: a systematic review. doi
  13. (2004). Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg doi
  14. (1993). Clinical studies in surgical journals-have we improved? Dis Colon Rectum doi
  15. (1983). Clinical Trials: A Practical Approach. Chapter 5: Methods of Randomisation. doi
  16. Complex interventions: how out of control can a randomised controlled trial be? doi
  17. (2006). Continuous monitoring of the performance of hip prostheses. J Bone Joint Surg Br.
  18. (2008). Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ doi
  19. (2009). Does the level of experience of residents affect outcomes of coronary artery bypass surgery? Ann Thorac Surg. doi
  20. (2004). Evidence-based practice requires evidence. doi
  21. Evidence-based practice: new techniques and technology.
  22. (2002). Experimental And Quasi-experimental Designs For Generalized Causal Inference. doi
  23. (2008). Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med doi
  24. (2004). for the Veterans Affairs Cooperative Studies Program. Open mesh versus laparoscopic mesh repair of inguinal hernia. doi
  25. (2009). Four years of teaching principles in clinical trials--a continuous evaluation of the postgraduate workshop for surgical investigators at the study center of the German Surgical Society. J Surg Educ. doi
  26. (2000). Framework for design and evaluation of complex interventions to improve health. BMJ.
  27. (2006). Healthrelated quality of life measurement in randomized clinical trials in surgical oncology. J Clin Oncol.
  28. (2009). How guidance on the use of interventional procedures is produced in different countries: an international survey. Int J Technol Assess Health Care. doi
  29. (2008). Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ doi
  30. (2002). Is it safe to train residents to perform cardiac surgery? Ann Thorac Surg. doi
  31. Is surgical science dead? The Excelsior Society lecture.
  32. (2004). Measuring the quality of surgical care: structure, process, or outcomes? J Am Coll Surg.
  33. Need for expertise based randomised controlled trials.
  34. Observational research, randomized trials, and two views of medical science. doi
  35. (2007). OMERACT: An international initiative to improve outcome measurement in rheumatology. Trials doi
  36. (2003). Perceptions of equipoise are crucial to trial participation: a qualitative study of men in the ProtecT study. Control Clin Trials. doi
  37. Perspectives of evidence-based surgery. doi
  38. Placebos that harm: sham surgery controls in clinical trials. doi
  39. (1994). Practical considerations in the management of large multi-institutional databases. Ann Thorac Surg
  40. (1996). prospective, single-blind comparison of laparoscopic versus small-incision cholecystectomy. doi
  41. (2004). Recent trends in the funding and utilization of NIH career development awards by surgical faculty. Surgery. doi
  42. Reporting methods of blinding in randomized trials assessing nonpharmacological treatments. doi
  43. Reporting of outcomes in orthopaedic randomized trials: does blind of outcome assessors matter? doi
  44. (2001). Statistical assessment of the learning curves of health technologies. Health Technol Assess
  45. (2004). Statistical evaluation of learning curve effects in surgical trials. Clin Trials doi
  46. (1978). Surgeon-related variables and the clinical trial. Lancet doi
  47. Surgical and oncology trials for rectal cancer: who will participate? doi
  48. (2006). Surgical vs nonoperative - 32 -treatment for lumbar disk herniation: the Spine Patient Outcomes Research Trial (SPORT): a randomized trial. JAMA doi
  49. (2008). Survival after bilateral versus single lung transplantation for patients with chronic obstructive pulmonary disease: a retrospective analysis of registry data. Lancet doi
  50. (1991). Symptomatic Carotid Endarterectomy Trial Collaborators. Beneficial effect of carotid endarterectomy in symptomatic patients with highgrade carotid stenosis. doi
  51. (2009). The challenges faced in the design, conduct and analysis of surgical randomised controlled trials. Trials doi
  52. The Clavien-Dindo Classification of Surgical Complications: Five-year Experience. doi
  53. The EC/IC Bypass Study Group. Failure of extracranial-intracranial arterial bypass to reduce the risk of ischemic stroke. Results of an international randomized trial. doi
  54. (2004). The future of surgical research PLoS Med. doi
  55. The measurement and monitoring of surgical adverse events. Health Technol Assess.
  56. The Southern Surgeons Club. A prospective analysis of 1518 laparoscopic cholecystectomies. doi
  57. (2007). The surgical learning curve for prostate cancer control after radical prostatectomy. doi
  58. TheIpswich Childbirth Study: 1. A randomised evaluation of two stage postpartum penned repair leaving the skin unsutured. doi
  59. (2005). Tolerance of uncertainty, extroversion, neuroticism and attitudes to randomized controlled trials among surgeons and physicians. Br J Surg. doi
  60. Treatment Trial Research Group. Patients at high risk of death after lung-volume-reduction surgery.
  61. (2000). Trials and fast changing technologies: the case for tracker studies. BMJ doi
  62. Use of Placebo Surgery in Controlled Trials of a Cellular-Based Therapy for Parkinson's Disease. doi
  63. (1993). Users' guides to the medical literature. I. How to get started. doi
  64. (2007). Using the literature to quantify the learning curve. nt J Technol Assess Health Care doi
  65. When are randomised trials unnecessary? Picking signal from noise. doi
  66. When is surgery research? Towards an operational definition of human research. doi
  67. Why we need observational studies to evaluate the effectiveness of health care BMJ. doi

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.