High Stakes Technical Skill Assessments in Surgery: Development, Implementation and Predicting Performance

A paradigm shift in surgical training from an apprenticeship to a competency based model requires valid assessment platforms to assess surgical competence. Several domains of competence are currently formally assessed, however technical skill, despite being the hallmark of a surgeon, is not directly or formally assessed upon completion of surgical training. The development, evaluation and implementation of objective tools to evaluate technical skill are essential, as surgical bodies mandate the documentation of competence. The first manuscript describes the first key step in developing a high stakes examination, using a Delphi methodology to develop a blueprint for the General Surgery Objective Structured Assessment of Technical Skill (GOSATS) exam. An essential component of a high stakes exam is setting a pass score. The second and third manuscripts address this aspect of high stakes exam development. The second manuscript applies and compares three standard setting methodologies to the worldâ s largest Objective Structured Assessment of Technical Skill (OSATS) database to set passing scores for the OSATS exam. General surgery residents (n=133) from the University of Toronto between 2002 and 2012 were then assigned a pass/fail status allowing for comparison of the consistency of the pass/fail decision across the methodologies. The third manuscript investigates the validity of these set pass scores by evaluating the predictive ability of the OSATS pass/fail status. The general surgery residents that had passed the OSATS were compared to the residents that had failed the OSATS in terms of their PGY2 and PGY4 technical skill. The results indicate that passing the OSATS exam predicts technical skill in the PGY2 and PGY4 year. The final manuscript is a validation study investigating the results of the first North American technical skills certification examination. In 2014 the Colorectal Objective Structured Assessment of Technical Skill (COSATS) was a mandatory component of the American Board of Colon and Rectal Surgeons certification process and represents the last paper in this dissertation. The results indicate that individuals who failed the COSATS, passed both the oral and written ABCRS exams, suggesting that the current certification process may fail to identify individuals with technical deficiencies.Ph.D

Post-discharge decisions: prolonged VTE prophylaxis for colorectal cancer surgery

The connection between venous thromboembolism (VTE) and cancer was first described in the 1860s by Armand Trousseau. Postoperative VTE remains one of the leading complications and causes of mortality in patients undergoing surgery for colorectal cancer. While there is evidence that continuation of VTE prophylaxis beyond the hospital stay can reduce overall VTE incidence in select groups of surgical patients, local hospital guidelines for prescribing post-discharge prophylactic medication to prevent VTE vary. Current guidelines from the American College of Chest Physicians (ACCP) recommend 30-day postoperative thromboprophylaxis for cancer patients undergoing major abdominal or pelvic surgery. This project aimed to develop an evidence-based protocol for post-discharge VTE prophylaxis in colorectal cancer patients undergoing surgical resection at St. Michael’s Hospital, Toronto. The project consisted of a literature review to define the indications for post-discharge VTE prophylaxis, a chart review of 100 patients with colorectal cancer undergoing colorectal surgery to determine compliance with best practice, and possible implementation of a new thromboprophylaxis protocol. After review of the 100 cases, 21 cases were excluded as the patients did not have a laparotomy or laparoscopic procedure, or were on anticoagulation therapy at time of/during admission. Only two of the 79 remaining patients (2.53%) were prescribed post-discharge thromboprophylaxis. This demonstrates a gap in the use of recommended guidelines for VTE prevention in cases of colorectal cancer resection. A protocol for implementing post-discharge VTE prophylaxis in cancer patients undergoing colorectal surgery at St. Michael’s Hospital, Toronto, was developed and will be discussed. </p