Latent Graph Representations for Critical View of Safety Assessment

Assessing the critical view of safety in laparoscopic cholecystectomy requires accurate identification and localization of key anatomical structures, reasoning about their geometric relationships to one another, and determining the quality of their exposure. Prior works have approached this task by including semantic segmentation as an intermediate step, using predicted segmentation masks to then predict the CVS. While these methods are effective, they rely on extremely expensive ground-truth segmentation annotations and tend to fail when the predicted segmentation is incorrect, limiting generalization. In this work, we propose a method for CVS prediction wherein we first represent a surgical image using a disentangled latent scene graph, then process this representation using a graph neural network. Our graph representations explicitly encode semantic information - object location, class information, geometric relations - to improve anatomy-driven reasoning, as well as visual features to retain differentiability and thereby provide robustness to semantic errors. Finally, to address annotation cost, we propose to train our method using only bounding box annotations, incorporating an auxiliary image reconstruction objective to learn fine-grained object boundaries. We show that our method not only outperforms several baseline methods when trained with bounding box annotations, but also scales effectively when trained with segmentation masks, maintaining state-of-the-art performance.Comment: 12 pages, 4 figure

The Identification and Classification of Flow Disruptions in the Operating Room during Laparoscopic Cholecystectomy and Open Hernia Repair Procedures

The operating room is one of the most complex work environments in healthcare; it is estimated that at least 7% of adverse events due to medical error occur in the operating room. Flow disruptions are events that cause a break in the primary surgical task, or the loss of any team member\u27s situational awareness. An empirical link between flow disruptions and surgical errors in the OR has been established; therefore, identifying and classifying the specific flow disruptions present during different types of procedures should facilitate the development of evidence-based interventions. The goal of this study was to identify and classify flow disruptions during laparoscopic cholecystectomy (camera-assisted gallbladder removal) and open inguinal and umbilical hernia repair procedures. Results of this study revealed seven categories of disruption that emerged inductively from the data collected. These were: communication, coordination, external/extraneous source, training/supervisory, equipment/supplies, patient factors, and environment. Though the average duration and disruption rate were similar for both types of procedure, the type of disruptions present during each were unique. One example of this includes the higher incidence of equipment related flow disruptions during laparoscopic cholesystechtomies, which is the more equipment intensive procedure of the two observed

Machine learning with different digital images classification in laparoscopic surgery

The evaluation of the effectiveness of the automatic computer diagnostic (ACD) systems developed based on two classifiers – HAAR features cascade and AdaBoost for the laparoscopic diagnostics of appendicitis and ovarian cysts in women with chronic pelvic pain is presented. The training of HAAR features cascade, and AdaBoost classifiers were performed with images/ frames, which have been extracted from video gained in laparoscopic diagnostics. Both gamma-corrected RGB and RGB converted into HSV frames were used for training. Descriptors were extracted from images with the method of Local Binary Pattern (LBP), which includes both data on color characteristics («modified color LBP» - MCLBP) and textural characteristics, which have been used later on for AdaBoost classifier training. Classification of test video images revealed that the highest recall for appendicitis diagnostics was achieved after training of AdaBoost with MCLBP descriptors extracted from RGB images – 0.708, and in the case of ovarian cysts diagnostics – for MCLBP gained from RGB images – 0.886. Developed AdaBoost-based ACD system achieved a 73.6% correct classification rate (accuracy) for appendicitis and 85.4% for ovarian cysts. The accuracy of the HAAR features classifier was highest in the case of ovarian cysts identification and achieved 0.653 (RGB) – 0.708 (HSV) values. It was concluded that the HAAR feature-based cascade classifier turned to be less effective when compared with the AdaBoost classifier trained with MCLBP descriptors. Ovarian cysts were better diagnosed when compared with appendicitis with the developed ACD

ПОРІВНЯЛЬНА ЕФЕКТИВНІСТЬ КЛАСИФІКАТОРІВ ЗОБРАЖЕНЬ ПІД ЧАС РОЗПІЗНАВАННЯ ЗОН ІНТЕРЕСУ ПРИ ЛАПАРОСКОПІЧНИХ ВТРУЧАННЯХ

Background. The purpose of the study is to evaluate the effectiveness of the automatic computer diagnostic (ACD) systems developed on the basis of two classifiers — HAAR features cascade and AdaBoost for the detection of appendicitis and metastatic damages of the liver. Materials and methods. For the classifiers training the images/frames, which have been cropped out from video gained in the course of laparoscopic diagnostics were used. Namely, RGB frames, and gamma-corrected RGB frames and converted into HSV have been explored. Also descriptors were extracted from images with the modified method of Local Binary Pattern (LBT), which includes data on color characteristics («modified color LBT» — MCLBT) and textural ones were used later on for AdaBoost classifier training. After cessation of training the tests were performed with the aim of the estimation of effectiveness of recognition. Test session images were different from those ones which have been used for training of the classifier. Results. The highest recall for appendicitis diagnostics was achieved after training of AdaBoost with MCLBT descriptors extracted from RGB images—0,745, and in case for metastatic damages diagnostics — 0,902. Hence developed AdaBoost based CAD system achieved 74,4 % correct classification rate (accuracy) for appendicitisc and 89,3 % for metastatic images. The accuracy of HAAR features classifier was highest in case of metastatic foci identification and achieved 0,672 (RGB) — 0,723 (HSV) values. Conclusions. Haar features based cascade classifier turned to be less effective when compared with AdaBoost classifier trained with MCLBT descriptors.В работе представлено сравнительные оценки эффективности систем автоматизированной компьютерной диагностики (АКД), разработанных на основе двух классификаторов — каскада дескрипторов Хаара и AdaBoost, во время лапароскопической диагностики аппендицита и метастазов печени. Для обучения применяли изображения, а также гамма-коррегированные и конвертированные в HSV шкалу RGB изображения, полученные при лапароскопической диагностике. Дескрипторы, которые использовали для обучения классификатора AdaBoost получали при помощи метода локального бинарного паттерна (ЛБП), который включал информационные показатели цвета («модифицированный цвет ЛБП» — MЦЛБП), а также показатели текстуры. После завершения обучения проводили тест оценки эффективности диагностики, при котором использовали изображения, неприменяемые для обучения. Наиболее высоким показатель полноты (recall) был при тестовой диагностике аппендицита с помощью классификатора AdaBoost в обучении применяли дескрипторы МЦЛБП, полученные при анализе RGB изображений — 0,745, а при диагностике метастазов печени — 0,902. Также корректность диагностики (accuracy) составила 74,4 % при диагностике аппендицита и 89,3 % при диагностике метастазов печени. Корректность диагностики при применении классификатора Хаара была наиболее высокой при диагностике метастазов печени и составила 0,672 при использовании RGB изображений и 0,723 — при обучении HSV изображениями. Диагностика с применением классификатора Хаара менее эффективна по сравнению с диагностикой, осуществляемой с применением классификатора AdaBoost, обучение последнего проводили с применением дескрипторов МЦЛБП.У роботі представлено порівняльне оцінювання ефективності систем автоматизованої комп'ютерної діагностики, розроблених на основі двох класифікаторів — каскаду дескрипторів Хаара та AdaBoost, під час лапароскопічної діагностики апендициту та метастазів печінки. Для навчання використовували зображення, а також гама-кореговані та конвертовані у HSV шкалу кольори RGB зображення, отримані під час лапароскопічної діагностики. Дескриптори, що використовували для навчання класифікатора AdaBoost отримували за допомогою методу локального бінарного патерну, який включав інформаційні показники кольору, а також показники текстури. Після завершення навчання проводили тест оцінювання ефективності діагностики при якому використовували зображення, що не застосовували для навчання. Найбільш високим показник повноти (recall) був при тестовій діагностиці апендициту за допомогою навчання класифікатора AdaBoost дескрипторами модифікованого кольору локального бінарного патерну, отриманими з RGB зображень, — 0,745, а під час діагностики метастазів печінки — 0,902. Також коректність діагностики (accuracy) склала 74,4 % під час діагностики апендициту та 89,3 % при діагностиці метастазів печінки. Коректність діагностики із застосуванням класифікатора Хаара була найбільш високою за умови діагностики метастазів печінки та склала 0,672 при використанні RGB зображень, 0,723 — при навчанні HSV зображеннями. Діагностика із застосуванням класифікатора Хаара є менш ефективною порівняно з діагностикою, що здійснювалась із застосуванням класифікатора AdaBoost, навчання якого здійснювали із застосуванням дескрипторів модифікованого кольору локального бінарного патерну

Perception and Orientation in Minimally Invasive Surgery

During the last two decades, we have seen a revolution in the way that we perform abdominal surgery with increased reliance on minimally invasive techniques. This paradigm shift has come at a rapid pace, with laparoscopic surgery now representing the gold standard for many surgical procedures and further minimisation of invasiveness being seen with the recent clinical introduction of novel techniques such as single-incision laparoscopic surgery and natural orifice translumenal endoscopic surgery. Despite the obvious benefits conferred on the patient in terms of morbidity, length of hospital stay and post-operative pain, this paradigm shift comes at a significantly higher demand on the surgeon, in terms of both perception and manual dexterity. The issues involved include degradation of sensory input to the operator compared to conventional open surgery owing to a loss of three-dimensional vision through the use of the two-dimensional operative interface, and decreased haptic feedback from the instruments. These changes have led to a much higher cognitive load on the surgeon and a greater risk of operator disorientation leading to potential surgical errors. This thesis represents a detailed investigation of disorientation in minimally invasive surgery. In this thesis, eye tracking methodology is identified as the method of choice for evaluating behavioural patterns during orientation. An analysis framework is proposed to profile orientation behaviour using eye tracking data validated in a laboratory model. This framework is used to characterise and quantify successful orientation strategies at critical stages of laparoscopic cholecystectomy and furthermore use these strategies to prove that focused teaching of this behaviour in novices can significantly increase performance in this task. Orientation strategies are then characterised for common clinical scenarios in natural orifice translumenal endoscopic surgery and the concept of image saliency is introduced to further investigate the importance of specific visual cues associated with effective orientation. Profiling of behavioural patterns is related to performance in orientation and implications on education and construction of smart surgical robots are drawn. Finally, a method for potentially decreasing operator disorientation is investigated in the form of endoscopic horizon stabilization in a simulated operative model for transgastric surgery. The major original contributions of this thesis include: Validation of a profiling methodology/framework to characterise orientation behaviour Identification of high performance orientation strategies in specific clinical scenarios including laparoscopic cholecystectomy and natural orifice translumenal endoscopic surgery Evaluation of the efficacy of teaching orientation strategies Evaluation of automatic endoscopic horizon stabilization in natural orifice translumenal endoscopic surgery The impact of the results presented in this thesis, as well as the potential for further high impact research is discussed in the context of both eye tracking as an evaluation tool in minimally invasive surgery as well as implementation of means to combat operator disorientation in a surgical platform. The work also provides further insight into the practical implementation of computer-assistance and technological innovation in future flexible access surgical platforms

A methodology for design and appraisal of surgical robotic systems

Surgical robotics is a growing discipline, continuously expanding with an influx of new ideas and research. However, it is important that the development of new devices take account of past mistakes and successes. A structured approach is necessary, as with proliferation of such research, there is a danger that these lessons will be obscured, resulting in the repetition of mistakes and wasted effort and energy. There are several research paths for surgical robotics, each with different risks and opportunities and different methodologies to reach a profitable outcome. The main emphasis of this paper is on a methodology for ‘applied research’ in surgical robotics. The methodology sets out a hierarchy of criteria consisting of three tiers, with the most important being the bottom tier and the least being the top tier. It is argued that a robotic system must adhere to these criteria in order to achieve acceptability. Recent commercial systems are reviewed against these criteria, and are found to conform up to at least the bottom and intermediate tiers, the most important first two tiers, and thus gain some acceptability. However, the lack of conformity to the criteria in the top tier, and the inability to conclusively prove increased clinical benefit, is shown to be hampering their potential in gaining wide establishment

Application of intraoperative quality assurance to laparoscopic total mesorectal excision surgery

Introduction: The role of laparoscopy in the surgical management of rectal cancer is debated. Randomised trials have reported contrasting results with inadequate specimens obtained in a minority of patients. The reasons behind these findings are unclear. Complex surgical interventions and human performance are prone to variation, which may account for outcome differences, but neither are robustly measured. Application of quality assurance (QA) to the intraoperative period could explore surgical performance and any relationship with subsequent outcomes. The overarching aim of this thesis is the promotion of oncological and patient safety through application of QA to laparoscopic TME surgery. Methods: Evidence synthesis of QA tools was obtained through a systematic review to identify reported objective laparoscopic total mesorectal excision (TME) assessment tools. Development of novel QA tools for laparoscopic TME was performed and applied and validated using case video from two multicentre randomised trials with reliability and validity of the laparoscopic TME performance tool (L-TMEpt) assessed. A multicentre randomised trial comparing 3D vs. 2D laparoscopic TME was performed incorporating objective performance analyses. Scores divided surgeons into quartiles and compared with histopathological and clinical endpoints. A novel intraoperative adverse event classification was developed and piloted. Results: 176 cases from 48 credentialed surgeons were analysed. L-TMEpt inter-rater, test-retest and internal consistency reliabilities were established. Substantial variation in surgical performance were seen. Scores were strongly associated with the number of intraoperative errors, plane of mesorectal dissection and short-term patient morbidity. Upper quartile surgeons obtained excellent results compared with the lower quartile (mesorectal fascia 93% vs. 59%, NNT 2.9, p=0.002; 30-day morbidity 23% vs. 48%, NNT 4, p=0.043). Conclusions: Intraoperative QA using assessment tools can objectively and reliably measure complex cancer interventions. Laparoscopic TME surgical performance assessment showed substantial variation which is strongly associated with clinical outcomes holding implications for surgical trial design and interpretation.Open Acces