Challenges in evaluating surgical innovation

Research on surgical interventions is associated with several methodological and practical challenges of which few, if any, apply only to surgery. However, surgical evaluation is especially demanding because many of these challenges coincide. In this report, the second of three on surgical innovation and evaluation, we discuss obstacles related to the study design of randomised controlled trials and non-randomised studies assessing surgical interventions. We also describe the issues related to the nature of surgical procedures—for example, their complexity, surgeon-related factors, and the range of outcomes. Although difficult, surgical evaluation is achievable and necessary. Solutions tailored to surgical research and a framework for generating evidence on which to base surgical practice are essential.The Balliol Colloquium has been supported by Ethicon UK with unrestricted educational grants and by the National Institute of Health Research Health Technology Assessment Programme. The Balliol Colloquium was administratively and financially supported by the Nuffield Department of Surgery at the University of Oxford and the Department of Surgery at McGill University. JAC holds a Medical Research Council UK special training fellowship. The University of Aberdeen’s Health Services Research Unit is core funded by the Chief Scientist Offi ce of the Scottish Government Health Directorates. IB is supported by a grant from the Société Française de Rhumatologie and Lavoisier Program (Ministère des Aff aires Etrangères et Européennes). PLE is a DPhil Candidate in Evidence-Based Health Care at Oxford University

Exploring methods for selection and integration of stakeholder views in the development of core outcome sets:a case study in reconstructive breast surgery

BACKGROUND: The development and use of core outcome sets (COSs) in trials may improve data synthesis and reduce outcome reporting bias. The selection of outcomes in COSs is informed by views of key stakeholders, yet little is known about the role and influence of different stakeholders’ views during COS development. We report an exploratory case study examining how stakeholder selection and incorporation of stakeholders’ views may influence the selection of outcomes for a COS in reconstructive breast surgery (RBS). We also make recommendations for future considerations. METHODS: Key stakeholder groups and subgroups were identified from the literature and expert opinion by the COS management group. They included health care professionals, subdivided by profession (breast and plastic surgeons, specialist nurses and psychologists) and patients, subdivided according to type of surgery received, timing of reconstruction, time since surgery and patient age. All participated in a survey in which they were asked to prioritise outcomes. Outcomes were prioritised using a 9-point scale from 1 (not important) to 9 (extremely important). The proportion of (1) all participants, ignoring stakeholder group (single heterogeneous panel analysis), (2) ‘professional’ and ‘patient’ groups separately (two heterogeneous panels), ignoring prespecified subgroups and (3) each participant subgroup separately (multiple homogeneous panel analysis) rating each item ‘extremely important’ was summarised and compared to explore how selection and integration of stakeholder views may influence outcome prioritisation. RESULTS: There were many overlaps between items rated as most important by all groups. Specific stakeholders, however, prioritised specific concerns and a broader range of outcomes were prioritised when the subgroups were considered separately. For example, two additional outcomes were prioritised when patient and professional groups were considered separately and eight additional outcomes were identified when the views of the individual subgroups were explored. In general, patient subgroups preferentially valued additional clinical outcomes, including unplanned surgery, whereas professional subgroups prioritised additional psychosocial issues including body image. CONCLUSION: Stakeholder groups value different outcomes. Selection of groups, therefore, is important. Our recommendations for robust and transparent stakeholder selection and integration of stakeholder views may aid future COS developers in the design and conduct of their studies and improve the validity and value of future COS