Evaluation of Wales Postgraduate Medical and Dental Education Deanery outcomes at core and higher general surgery before and after national reconfiguration, enhanced selection, and Joint Committee on Surgical Training defined curricular standards

This thesis examines contemporary outcomes of surgical training in Wales and the UK. The hypotheses tested were: Core Surgical Training (CST) outcome is related to specific curricular defined goals, and themed focused CST rotations improve success at National Training Number (NTN)appointment; CST rotations including rural placements provide training comparable with non-rural placements; General Surgery (GS) Certificate of Completion of Training (CCT) curricular guidelines require focused appraisal and rotation planning; GS Higher Surgical Trainee (HST) indicative procedure targets are not in keeping with competence achievement determined by Procedural Based Assessment (PBA); Dedicated Emergency General Surgery (EGS) modules enhance HST training experience; H-Indices are a valid measure of GS consultant academic productivity and identify training research opportunity. Successful ST3 NTN appointment improved from 5.3 to 33.3% (p=0.005) following CST [OR 4.789 (1.666 - 13.763), p=0.004] and is independently associated with success. ST3 appointment was similar irrespective of rural or non-rural CST rotational placement (18.1 vs. 22.1%, p=0.695). Of the 155 UK GS HST CCTs awarded in 2013, global operative log book and academic achievements varied widely, with two-thirds of trainees achieving elective operative targets, but only half the requisite experience in EGS, and 5% nonoperative targets. Wales’ HSTs level 4 GS operative competencies varied 4- fold, ranging from 0.76 to 3.4 times national targets. EGS modular training introduction delivered a high volume of index EGS procedures and higher rates of PBA completion when compared with controls. H-indices were a robust measure of surgeons’ academic activity (p<0.001)

Using keywords to predict the need for an audiogram: an analysis of referral letters using logistic regression

Background Otolaryngology clinics are often booked without considering the distribution of work for doctors and audiologists. This causes inefficiencies of time and human resources. This may be due to clinics being booked before referrals have been triaged to identify whether a hearing test, known as a pure tone audiogram (PTA), is indicated. A model that can predict the need for PTA without clinician-led triage could be useful to address these booking issues. Objective To establish if it is possible to predict whether a referred patient requires a PTA based on occurrences of words in the referral letter. Method Binary logistic regression analysis of 500 letters of referral for otolaryngology outpatients. The derived model was then tested on 50 referral letters. All the referral letters were reviewed by clinicians and classified according to whether or not a PTA would be required. Results The regression model correctly predicted requirement for a PTA in 42 of 50 referral letters (84%), with a sensitivity of 91% and specificity of 82%. Conclusion The model is able to predict requirement for a PTA from referral letters with a high degree of accuracy. This method may have a role in assisting administrative/clerical staff or nonspecialist clinicians to book appropriate ear, nose and throat (ENT) clinic appointments, with or without a PTA. As a result, workload would be distributed more evenly, through the clinic for both otolaryngologists and audiologists, increasing efficiency

Economic cost-utility analysis of stage-directed oesophageal cancer treatment

Introduction Oesophageal cancer (OC) treatment is guided by radiological diagnostic stage1, with prognosis worsening as stage advances2. Potentially curative treatment is possible in 30–40% of patients2,3. Treatments include definitive chemoradiotherapy or surgery, with or without neoadjuvant therapy4, and patients not considered suitable for curative treatment receive palliative treatments or Best Supportive Care (BSC)2,5. Clinical effectiveness of treatments can be estimated in terms of Quality-Adjusted Life Years (QALYs) to price a treatment’s cost-effectiveness6. The aim of this study was to estimate the cost-utility of curative treatment related to OC stage compared with BSC. The primary hypothesis was that OCs of earlier stage would prove cheaper to treat in fiscal terms than OCs of more advanced later stage. Methods Consecutive patients undergoing surgical treatment for OC diagnosed according to established protocols with curative intent within a regional cancer network from 2010 to 2020 were included in the analysis. The cost of 1-year’s treatment from referral was calculated based on current management standards. Primary outcome was overall survival (OS). Detailed methods can be found in Supplementary methods. Results 365 patients (median age 65 years (range 38–80), 308 male, 57 female, 263 neoadjuvant therapy) who underwent surgical treatment with curative intent for OC were included. Based on pathological and intraoperative assessment, 111 (30.4%) were stage I, 65 (17.8%) stage II, 118 (32.3%) stage III and 71 (19.5%) were analysed as stage IV. Of these, 331 had adenocarcinoma, 32 squamous cell carcinoma and two high-grade dysplasia. Median follow-up was 36 (interquartile range (i.q.r.) 34.9–39.0) months and median OS was 42.9 (95% c.i. 35.6 to 53.2) months with an average cost of the first year’s treatment of €30 916. This resulted in a QALY-adjusted survival of 34.3 months, with cost per QALY of €10 817. In patients who underwent curatively intended surgery, median survival in the patients receiving neoadjuvant chemotherapy followed by surgery (CS) was 35.8 (95% c.i. 26.5 to 45.1) months compared with 45.6 (95% c.i. 37.7 to 46.7) months in the patients receiving neoadjuvant chemoradiotherapy followed by surgery (CRS) and 50.8 (95% c.i. 39.4 to 47.2) months in patients receiving surgery (S) alone. The QALY-adjusted survival was 28.6 months in the CS cohort, compared with 36.5 in the CRS cohort and 40.6 in the S cohort. The cost per QALY for CS was €14 448, CRS €13 040 and S €5276. The CS cohort had a significantly lower proportion of patients with pTNM stage I and II disease (28.6%) compared with the CRS and S cohorts (66.7 and 69.6% respectively, P < 0.001). Data relating to QALY-adjusted survival and the cost per QALY, stratified by tumour stage, can be found in Table 1. The cost analysis of treating OC related to TNM stage and treatment modality can be found in Fig. 1. Median OS for patients receiving BSC reported in the literature is around 4 months7, with a Health State Utility Value (HSUV) of 0.56, equating to a QALY-adjusted survival of 2.24 months and a cost per QALY of €70 463

Prospective cohort study of haptic virtual reality laparoscopic appendicectomy learning curve trajectory

Background: Simulation training is strongly advocated by 24/7 risk-rich professions because swift learning curve inflection point attainment delivers earlier competence; the left-shift effect. The aim of this study was to determine the value of haptic laparoscopic virtual reality simulation, by iterative benchmark exercise (n = 8), before simulated laparoscopic appendicectomy (SLA); the hypothesis was that favorable benchmark learning curve trajectories would be associated with improved SLA competence when compared with consultant expert performance. Methods: A 28-trainee cohort completed 1349 Laparoscopic Haptic Virtual Reality Skills (LHVRS) tasks, during which 19 ergonomic variables were assessed by virtual interface, including force feedback (Surgicalscience.com), before 153 SLAs. Primary outcome measure was SLA composite competence score related to six consultant trainer experts. Results: Of the eight LHVRS tasks, the three with the steepest learning curve trajectories correlated with better median overall SLA competence scores, namely tissue grasping/lifting (rho = 0.362, P = .049), fine dissection (rho = 0.388, P = .028), and camera navigation (rho = 0.518, P = .007); fine dissection was the only haptic laparoscopic virtual reality simulation task that predicted a SLA score within a Youden index defined, 70% of the consultant expert level (area under curve [AUC] = 0.803, P = .028). A significant SLA learning curve emerged, with a learning curve trajectory inflection point at the fourth SLA attempt (first SLA 30.5% versus fourth SLA score 76.0%, gradient 76°, P = .010). Conclusion: Learning curve trajectory can be measured, influenced, and accelerated significantly; a pronounced left-shift effect, with translational potential for enhanced shorter training time and improved patient safety

Relative value of adapted novel bibliometrics in evaluating surgical academic impact and reach

Background The Hirsch index, often used to assess research impact, suffers from questionable validity within the context of General Surgery, and consequently adapted bibliometrics and altmetrics have emerged, including the r-index, m-index, g-index and i10-index. This study aimed to assess the relative value of these novel bibliometrics in a single UK Deanery General Surgical Consultant cohort. Method Five indices (h, r, m, g and i10) and altmetric scores (AS) were calculated for 151 general surgical consultants in a UK Deanery. Indices and AS were calculated from publication data via the Scopus search engine with assessment of construct validity and reliability. Results The median number of publications, h-index, r-index, m-index, g-index and i10-index were 13 (range 0–389), 5 (range 0–63), 5.2 (range 0–64.8), 0.33 (range 0–1.5), 10 (range 0–125) and 4 (range 0–245), respectively. Correlation coefficients of r-index, m-index, g-index and i10-index with h-index were 0.913 (p < 0.001), 0.716 (p < 0.001), 0.961 (p < 0.001) and 0.939 (p < 0.001), respectively. Significant variance was observed when the cohort was ranked by individual bibliometric measures; the median ranking shifts were: r-index − 2 (− 46 to + 23); m-index − 6.5 (− 53 to + 22); g-index − 0.5 (− 24 to + 13); and i10-index 0 (− 8 to + 11), respectively (p < 0.001). The median altmetric score and AS index were 0 (range 0–225.5) and 1 (range 0–10), respectively; AS index correlated strongly with h-index (correlation coefficient 0.390, p < 0.001). Conclusions Adapted bibliometric indices appear to be equally valid measures of evaluating academic productivity, impact and reach

Egalitarianism in surgical training: let equity prevail

This study aimed to quantify core surgical trainee (CST) differential attainment (DA) related to three cohorts; white UK graduate (White UKG) versus black and minority ethnic UKG (BME UKG) versus international medical graduates (IMGs). The primary outcome measures were annual review of competence progression (ARCP) outcome, intercollegiate Membership of the Royal College of Surgeons (iMRCS) examination pass and national training number (NTN) selection. Intercollegiate Surgical Curriculum Programme (ISCP) portfolios of 264 consecutive CSTs (2010–2017, 168 white UKG, 66 BME UKG, 30 IMG) from a single UK regional post graduate medical region (Wales) were examined. Data collected prospectively over an 8-year time period was analysed retrospectively. ARCP outcomes were similar irrespective of ethnicity or nationality (ARCP outcome 1, white UKG 60.7% vs BME UKG 62.1% vs IMG 53.3%, p=0.395). iMRCS pass rates for white UKG vs BME UKG vs IMG were 71.4% vs 71.2% vs 50.0% (p=0.042), respectively. NTN success rates for white UKG vs BME UKG vs IMG were 36.9% vs 36.4% vs 6.7% (p=0.023), respectively. On multivariable analysis, operative experience (OR 1.002, 95% CI 1.001 to 1.004, p=0.004), bootcamp attendance (OR 2.615, 95% CI 1.403 to 4.871, p=0.002), and UKG (OR 7.081, 95% CI 1.556 to 32.230, p=0.011), were associated with NTN appointment. Although outcomes related to BME DA were equitable, important DA variation was apparent among IMGs, with iMRCS pass 21.4% lower and NTN success sixfold less likely than UKG. Targeted counter measures are required to let equity prevail in UK CST programmes

Variations in competencies needed to complete surgical training

Background This study aimed to analyse the degree of relative variation in specialty‐specific competencies required for certification of completion of training (CCT) by the UK Joint Committee on Surgical Training. Methods Regulatory body guidance relating to operative and non‐operative surgical skill competencies required for CCT were analysed and compared. Results Wide interspecialty variation was demonstrated in the required minimum number of logbook cases (median 1201 (range 60–2100)), indexed operations (13 (5–55)), procedure‐based assessments (18 (7–60)), publications (2 (0–4)), communications to learned associations (0 (0–6)) and audits (4 (1–6)). Mandatory courses across multiple specialties included: Training the Trainers (10 of 10 specialties), Advanced Trauma Life Support (6 of 10), Good Clinical Practice (9 of 10) and Research Methodologies (8 of 10), although no common accord was evident. Discussion Certification guidelines for completion of surgical training were inconsistent, with metrics related to minimum operative caseload and academic reach having wide variation

Summative supervisor reporting: a quality performance perspective

Objective This study aimed to quality assure Assigned Educational Supervisor (AES) reports, using UK Joint Committee on Surgical Training objective criteria, to evaluate contribution to Annual Review of Competence Progression. Design Consecutive 145 AES reports from 75 trainers regarding 68 Core Surgical Trainees were assessed from 9 hospitals (2 Tertiary centers [77 reports], 7 District General Hospitals [68 reports]). Reports were assessed by independent assessors based on free text related to performance mapped to curricular objectives, operative logbooks, and Clinical Supervisor reports, and overall summary grades assigned ranging from development required, adequate, good to excellent. Setting A core surgical training program serving a single UK (Wales) deanery. Participants Sixty-eight consecutively appointed core surgical trainees and 75 consultant surgeon trainers. Results Summary grades of adequate or above were achieved in 101 of 145 (69.7%) reports. Trainees’ objective setting meetings were completed within 6 weeks of starting placements in 124 of 145 (85.5%). The proportions of AES reports containing free text commentary on curricular objectives, portfolio objectives, and operative logbook development were 128 of 145, 123 of 145, and 55 of 145, respectively. AES report quality was not associated with hospital status, subspecialty, or trainee grade. Female trainers were significantly more likely to provide reports graded as Good or Excellent compared with their male colleagues (7 of 12 vs. 27 of 133, χ2 (2) = 9.389, p = 0.009). AES reports for male trainees were significantly more likely to be rated as further development required (40 of 85, 47.1%) when compared with female trainees (4 of 32, 12.5%, p = 0.007). Conclusions Three in ten AES reports were insufficient to contribute to objective Annual Review of Competence Progression outcomes and a gender gap was apparent related to engagement. AES trainers should provide more focus if this summative tool is to be an effective career progression metric