Systematic review and narrative synthesis of surgeons’ perception of postoperative outcomes and risk

Background The accuracy with which surgeons can predict outcomes following surgery has not been explored in a systematic way. The aim of this review was to determine how accurately a surgeon's ‘gut feeling’ or perception of risk correlates with patient outcomes and available risk scoring systems. Methods A systematic review was undertaken in accordance with PRISMA guidelines. A narrative synthesis was performed in accordance with the Guidance on the Conduct of Narrative Synthesis In Systematic Reviews. Studies comparing surgeons' preoperative or postoperative assessment of patient outcomes were included. Studies that made comparisons with risk scoring tools were also included. Outcomes evaluated were postoperative mortality, general and operation‐specific morbidity and long‐term outcomes. Results Twenty‐seven studies comprising 20 898 patients undergoing general, gastrointestinal, cardiothoracic, orthopaedic, vascular, urology, endocrine and neurosurgical operations were included. Surgeons consistently overpredicted mortality rates and were outperformed by existing risk scoring tools in six of seven studies comparing area under receiver operating characteristic (ROC) curves (AUC). Surgeons' prediction of general morbidity was good, and was equivalent to, or better than, pre‐existing risk prediction models. Long‐term outcomes were poorly predicted by surgeons, with AUC values ranging from 0·51 to 0·75. Four of five studies found postoperative risk estimates to be more accurate than those made before surgery. Conclusion Surgeons consistently overestimate mortality risk and are outperformed by pre‐existing tools; prediction of longer‐term outcomes is also poor. Surgeons should consider the use of risk prediction tools when available to inform clinical decision‐making. Introduction Surgical procedures all carry associated risks. It is therefore important that surgeons are able to make accurate predictions of potential benefit and risk, including immediate mortality and morbidity, as well as long‐term outcomes, to enable balanced decision‐making and fully informed consent. Risks can also be estimated after surgery, based on additional perioperative and intraoperative data, which allows contemporary prediction of outcome. There are numerous risk prediction models that enable the surgeon to quantify risk based on measurable parameters1-5. However, there are inherent limitations in using a generalized risk prediction model, which may not include clinical data pertinent to the individual case in question, leading to variability in model accuracy6-10. As a result, risk prediction tools are generally used in tandem with the surgeon's ‘gut feeling’ of overall risk and anticipated outcome (‘clinical gestalt’). Several disparate factors influence surgeons' perception of outcome: patient factors, such as their perceived fitness, their pathology and planned procedure; setting factors, such as the experience of other members of staff; and surgeon factors, such as clinical knowledge, operative skill, previous significant surgical complications, and inclinations and attitudes11-13. Anticipating surgical risk is subject to multiple biases, which make it challenging. These include the natural tendency toward anecdotal recall and the availability heuristic (the likelihood of making a decision based on how easily the topic or examples come to mind)14, 15. Some studies16-18 support the accuracy and reproducibility of surgeons' predictions, whereas others19-22 demonstrate less favourable results. The complexity of synthesizing risk perceptions is significant and incompletely understood23, 24. The accuracy of surgeons' prediction has not been explored previously in a systematic manner. The aim of this review was thus to determine, from the available evidence, whether a surgeon's gut feeling or perception of risk correlates with postoperative outcomes, and to compare this prediction with currently available risk scoring systems, where available. Methods This systematic review was undertaken in accordance with the PRISMA guidelines25, 26. MEDLINE (via PubMed), Embase, the Cochrane Library Database, and the Cochrane Collaboration Central Register of Controlled Clinical Trials were searched with no date or language restrictions, with the last search date on 9 July 2018. The search term used was (‘Surgeons’[Mesh] OR ‘General Surgery/manpower*’ [MeSH]) AND (‘perception’ OR ‘intuition’ OR ‘predict*’ OR ‘decision making’ [mesh]). There was no restriction on publication type. This search was complemented by an exhaustive review of the bibliography of key articles, and also by using the Related Articles function in PubMed of included papers. Results were restricted to human research published in English. Inclusion and exclusion criteria All studies of patients undergoing surgery in which a preoperative or postoperative surgeon assessment (or proxy assessment) of a postoperative outcome was performed were included. This included articles that reported general risk (such as mortality) or a surgery‐specific risk (for example anastomotic leakage). Studies that made comparisons with established risk scoring tools were also included. Papers or abstracts in English, or non‐English papers with an English abstract, were included. Papers describing the risk assessment of ‘theoretical’ cases, or patient vignettes in a situation distant from clinical practice (such as a conference), were excluded, as were studies in which surgeons' assessment of risk was compared with an established risk scoring tool, without data on actual patient outcome. Data extraction and assessment of study quality Three authors independently extracted data and assessed the methodological quality of the studies, with all data extraction independently checked by the senior author. The following baseline data were extracted from each study: first author, year of publication, data collection period, geographical location, study design and type (single or multiple centres, number of surgeons involved in risk estimation, whether consecutive patients were enrolled), surgical specialty, whether other risk scoring systems were used for comparison and, if so, whether the assessor was blinded to this result. Data extracted regarding the assessment of risk included: risk outcome assessed; timing of risk estimation (preoperative or postoperative); type of risk assessment by surgeons (qualitative, quantitative, continuous scale such as a visual analogue scale (VAS), or composite score); absolute value of risk event predicted by surgeon and by scoring system; absolute value of risk occurrence rate; summary data on outcome reported, including area under the curve (AUC) of receiver operating characteristic (ROC) curves, observed : expected (O : E) or predicted : observed (P : O) ratios, or any other summary data. When data were available, AUCs were extracted with their 95 per cent confidence intervals. AUCs greater than 0·9 were considered as indicating high performance, 0·7–0·9 as moderate performance, 0·5–0·7 as low performance, and less than 0·5 as indicating risk assessment no better than chance alone27, 28. Risk predictions made by pre‐existing tools, such as the Physiological and Operative Severity Score for the enumeration of Mortality and morbidity (POSSUM)1, Portsmouth‐POSSUM (P‐POSSUM)4 or Continuous Improvement in Cardiac Surgery Program (CICSP)5, were compared with outcome when given. Internal prediction models, where authors would derive significant predictive co‐variables from their data set and assess the accuracy of these co‐variables within the same data set, were not evaluated as they lacked validity. Study quality was assessed using the Newcastle–Ottawa (NO) score29, 30. The NO score assigns points based on: the quality of patient selection (maximum 4 points); comparability of the cohort (maximum 2 points); and outcome assessment (maximum 3 points). Studies that scored 6 points or more were considered to be of higher quality. Outcome measures The following outcome measures were defined a priori and refined during data extraction: postoperative mortality (usually defined as 30 days after surgery); postoperative general morbidity (usually defined as 30 days after surgery); postoperative procedure‐specific morbidity; and long‐term outcome (typically operation‐specific). Further comparative analyses of outcomes included comparison of preoperative and postoperative predictions, and of predictions made by consultants and surgical trainees. Narrative synthesis Given the marked heterogeneity in study design, patient population included, method of assessing risk and outcomes assessed, meta‐analysis was deemed not appropriate. A narrative synthesis was therefore performed according to the Guidance on the Conduct of Narrative Synthesis In Systematic Reviews31. Three authors systematically summarized each article using bullet points to document key aspects of each study, focusing particularly on methods used and results obtained. The validity and certainty of the results were noted (whether appropriate statistical comparisons were used and, if so, their effect size and significance). The senior author identified and grouped common themes, divided larger themes into subthemes, tabulated a combined summary of the paper, and synthesized a common rubric for each theme. Consolidated reviewers' comments can be found in Table S1 (supporting information). Results A total of 584 articles were identified from the literature search, of which 48 were retrieved for evaluation. Papers were excluded on the basis of being duplicates (1) and being irrelevant based on the title (497) and abstract (38) (Fig. 1). Twenty‐seven studies16-24, 32-49 comprising 20 898 patients met the inclusion criteria and were included in the narrative synthesis (Appendix S1, supporting information)

Single versus dual antiplatelet therapy following peripheral arterial endovascular intervention for chronic limb threatening ischaemia:retrospective cohort study

ObjectivesAntiplatelet therapy following peripheral arterial endovascular intervention lacks high quality evidence to guide practice. The aim of this study was to assess the effect of three months of dual antiplatelet therapy on amputation-free survival following peripheral arterial endovascular intervention in patients with chronic limb threatening ischemia.MethodsA retrospective review of symptomatic patients undergoing primary peripheral arterial endovascular intervention over a seven-year period was performed. The primary outcome measure was amputation-free survival. A sample size calculation based on previous cohort studies suggested that 629 limbs would be required to show a difference between single and dual therapy. Kaplan-Meier estimates and multivariate logistic regression analysis of recorded baseline characteristics was performed to determine predictors of amputation-free survival. Dual antiplatelet therapy was routinely given for 3 months.Results754 limbs were treated with primary angioplasty and/or stenting over a 7-year period, 508 of these for chronic limb threatening ischemia. There was no difference in unadjusted amputation-free survival between patients with chronic limb threatening ischaemia taking single vs. dual antiplatelet therapy (69% vs. 74% respectively Log rank Chi2 = 0.1, p = .72). After adjusting for confounders, at 1 year there was also no significant difference in amputation-free survival between patients taking single vs. dual antiplatelet therapy [OR 0.8, 95% CI 0.5-1.2, p = .3]. There was no difference in rates of major bleeding between single and dual antiplatelet therapy.ConclusionsThere was no clear evidence of reduced amputation-free survival in patients with chronic limb threatening ischemia undergoing peripheral arterial endovascular intervention being treated with dual antiplatelet therapy for 3 months. This is at odds with other retrospective case series and highlights the limitations in basing clinical practice on such data. There is a need for an adequately powered, independent randomised trial to definitively answer the question

Exploring patients’ experiences of analgesia after major lower limb amputation: A qualitative study

OBJECTIVES: To explore patient experiences, understanding and perceptions of analgesia following major lower limb amputation. DESIGN: Qualitative interview study, conducted as part of a randomised controlled feasibility trial. SETTING: Participants were recruited from two general hospitals in South Wales. PARTICIPANTS: Interview participants were patients enrolled in PLACEMENT (Perineural Local Anaesthetic Catheter aftEr Major lowEr limb amputatioN Trial): a randomised controlled feasibility trial comparing the use of perineural catheter (PNC) versus standard care for postoperative pain relief following major lower limb amputation. PLACEMENT participants who completed 5-day postoperative follow-up, were able and willing to participate in a face-to-face interview, and had consented to be contacted, were eligible to take part in the qualitative study. A total of 20 interviews were conducted with 14 participants: 10 male and 4 female. METHODS: Semi-structured, face-to-face interviews were conducted with participants over two time points: (1) up to 1 month and (2) at least 6 months following amputation. Interviews were audio-recorded, transcribed verbatim and analysed using a framework approach. RESULTS: Interviews revealed unanticipated benefits of PNC usage for postoperative pain relief. Participants valued the localised and continuous nature of this mode of analgesia in comparison to opioids. Concerns about opioid dependence and side effects of pain relief medication were raised by participants in both treatment groups, with some reporting trying to limit their intake of analgesics. CONCLUSIONS: Findings suggest routine placement of a PNC following major lower limb amputation could reduce postoperative pain, particularly for patient groups at risk of postoperative delirium. This method of analgesic delivery also has the potential to reduce preoperative anxiety, alleviate the burden of pain management and minimise opioid use. Future research could further examine the comparison between patient-controlled analgesia and continuous analgesia in relation to patient anxiety and satisfaction with pain management. TRIAL REGISTRATION NUMBER: ISRCTN: 85710690; EudraCT: 2016-003544-37

PrEdiction of Risk and Communication of outcomE followIng major lower limb amputation – a collaboratiVE study (PERCEIVE): Protocol for the PERCEIVE qualitative study

INTRODUCTION: Deciding whether to proceed with a major lower limb amputation is life-changing and complex, and it is crucial that the right decision is made at the right time. However, medical specialists are known to poorly predict risk when assessing patients for major surgery, and there is little guidance and research regarding decisions about amputation. The process of shared decision-making between doctors and patients during surgical consultations is also little understood. Therefore, the aim of this study is to analyse in depth the communication, consent, risk prediction and decision-making process in relation to major lower limb amputation. METHODS AND ANALYSIS: Consultations between patients and surgeons at which major lower limb amputation is discussed will be audio-recorded for 10–15 patients. Semi-structured follow-up interviews with patients (and relatives/carers) will then be conducted at two time points: as soon as possible/appropriate after a decision has been reached regarding surgery, and approximately 6 months later. Semi-structured interviews will also be conducted with 10–15 healthcare professionals working in the UK National Health Service (NHS) involved in amputation decision-making. This will include surgeons, anaesthetists and specialist physiotherapists at 2–4 NHS Health Boards/Trusts in Wales and England. Discourse analysis will be used to analyse the recorded consultations; interviews will be analysed thematically. Finally, workshops will be held with patients and healthcare professionals to help synthesise and interpret findings. ETHICS AND DISSEMINATION: The study has been approved by Wales REC 7 (20/WA/0351). Study findings will be published in international peer-reviewed journal(s) and presented at national and international scientific meetings. Findings will also be disseminated to a wide NHS and lay audience via presentations at meetings and written summaries for key stakeholder groups

The PERCEIVE quantitative study: PrEdiction of Risk and Communication of outcome following major lower limb amputation: protocol for a collaboratiVE study

BACKGROUND: Accurate prediction of outcomes following surgery with high morbidity and mortality rates is essential for informed shared decision-making between patients and clinicians. It is unknown how accurately healthcare professionals predict outcomes following major lower-limb amputation (MLLA). Several MLLA outcome-prediction tools have been developed. These could be valuable in clinical practice, but most require validation in independent cohorts before routine clinical use can be recommended. The primary aim of this study is to evaluate the accuracy of healthcare professionals’ predictions of outcomes in adult patients undergoing MLLA for complications of chronic limb-threatening ischaemia (CLTI) or diabetes. Secondary aims include the validation of existing outcome-prediction tools. METHOD: This study is an international, multicentre prospective observational study including adult patients undergoing a primary MLLA for CLTI or diabetes. Healthcare professionals’ accuracy in predicting outcomes at 30-days (death, morbidity and MLLA revision) and 1-year (death, MLLA revision and ambulation) will be evaluated. Sixteen existing outcome-prediction tools specific to MLLA will be examined for validity. Data collection began on 1 October 2020; the end of follow-up will be 1 May 2022. The C-statistic, Hosmer–Lemeshow test, reclassification tables and Brier score will be used to evaluate the predictive performance of healthcare professionals and prediction tools, respectively. STUDY REGISTRATION AND DISSEMINATION: This study will be registered locally at each centre in accordance with local policies before commencing data collection, overseen by local clinician leads. Results will be disseminated to all centres, and any subsequent presentation(s) and/or publication(s) will follow a collaborative co-authorship model

Perineural Local Anaesthetic Catheter aftEr Major lowEr limb amputatioN Trial (PLACEMENT):results from a randomised controlled feasibility trial

Objectives To determine the feasibility of undertaking a randomised controlled effectiveness trial evaluating the use of a perineural catheter (PNC) after major lower limb amputation with postoperative pain as the primary outcome. Design Randomised controlled feasibility trial. Setting Two vascular Centres in South Wales, UK. Participants 50 patients scheduled for major lower limb amputation (below or above knee) for complications of peripheral vascular disease. Interventions The treatment arm received a PNC placed adjacent to the sciatic or tibial nerve at the time of surgery, with continuous infusion of levobupivacaine hydrochloride 0.125% for up to 5 days. The control arm received neither local anaesthetic nor PNC. Both arms received usual perioperative anaesthesia and postoperative analgesia. Primary and secondary outcome measures The primary outcomes were the proportion of eligible patients who were randomised and the proportion of recruited patients who provided primary effectiveness outcome data. Secondary outcomes were: the proportion of recruited patients reaching 2 and 6 month follow-up and supplying pain data; identification of key cost drivers; development of an economic analysis framework for a future effectiveness trial; identification of barriers to recruitment and site set-up; and identification of the best way to measure postoperative pain. Results Seventy-six of 103 screened patients were deemed eligible over a 10 month period. Fifty (64.5%) of these patients were randomised, with one excluded in the perioperative period. Forty-five (91.3%) of 49 recruited patients provided enough pain scores on a 4-point verbal rating scale to allow primary effectiveness outcome evaluation. Attrition rates were high; 18 patients supplied data at 6 month follow-up. Costs were dominated by length of hospital stay. Patients and healthcare professionals reported that trial processes were acceptable. Conclusions Recruitment of patients into a trial comparing PNC use to usual care after major lower limb amputation with postoperative pain measured on a 4-point verbal rating scale is feasible. Evaluation of longer-term symptoms is difficult