Concept Graph Neural Networks for Surgical Video Understanding

We constantly integrate our knowledge and understanding of the world to enhance our interpretation of what we see. This ability is crucial in application domains which entail reasoning about multiple entities and concepts, such as AI-augmented surgery. In this paper, we propose a novel way of integrating conceptual knowledge into temporal analysis tasks via temporal concept graph networks. In the proposed networks, a global knowledge graph is incorporated into the temporal analysis of surgical instances, learning the meaning of concepts and relations as they apply to the data. We demonstrate our results in surgical video data for tasks such as verification of critical view of safety, as well as estimation of Parkland grading scale. The results show that our method improves the recognition and detection of complex benchmarks as well as enables other analytic applications of interest

SAGES consensus recommendations on surgical video data use, structure, and exploration (for research in artificial intelligence, clinical quality improvement, and surgical education)

BACKGROUND: Surgery generates a vast amount of data from each procedure. Particularly video data provides significant value for surgical research, clinical outcome assessment, quality control, and education. The data lifecycle is influenced by various factors, including data structure, acquisition, storage, and sharing; data use and exploration, and finally data governance, which encompasses all ethical and legal regulations associated with the data. There is a universal need among stakeholders in surgical data science to establish standardized frameworks that address all aspects of this lifecycle to ensure data quality and purpose. METHODS: Working groups were formed, among 48 representatives from academia and industry, including clinicians, computer scientists and industry representatives. These working groups focused on: Data Use, Data Structure, Data Exploration, and Data Governance. After working group and panel discussions, a modified Delphi process was conducted. RESULTS: The resulting Delphi consensus provides conceptualized and structured recommendations for each domain related to surgical video data. We identified the key stakeholders within the data lifecycle and formulated comprehensive, easily understandable, and widely applicable guidelines for data utilization. Standardization of data structure should encompass format and quality, data sources, documentation, metadata, and account for biases within the data. To foster scientific data exploration, datasets should reflect diversity and remain adaptable to future applications. Data governance must be transparent to all stakeholders, addressing legal and ethical considerations surrounding the data. CONCLUSION: This consensus presents essential recommendations around the generation of standardized and diverse surgical video databanks, accounting for multiple stakeholders involved in data generation and use throughout its lifecycle. Following the SAGES annotation framework, we lay the foundation for standardization of data use, structure, and exploration. A detailed exploration of requirements for adequate data governance will follow

Relatório de estágio em farmácia comunitária

Relatório de estágio realizado no âmbito do Mestrado Integrado em Ciências Farmacêuticas, apresentado à Faculdade de Farmácia da Universidade de Coimbr

Do the costs of robotic surgery present an insurmountable obstacle? A narrative review

With increasing market size and rising demand, the question arises whether the high cost impedes accessibility to robotic surgery. Despite all the apparent advantages robotic surgery offers to surgeons and patients, it is imperative for healthcare providers to weigh the insufficiently documented evidence for robotics against the exorbitant price. Aside from the high acquisition cost of robotic systems, the cost of instruments and accessories, maintenance, as well as the need for training, and the impact on procedural dynamics in the operating room factor into any cost–utility analysis. However, current perspectives provide an insufficient overview of available systems and their cost. And the lack of transparency and incomplete information provided by manufacturers impose a significant challenge to informed decision-making. This article gives a short overview of the cost of robotic surgery, what additional costs to consider, where to obtain information, and attempts to elaborate on the question of whether cost impedes the worldwide establishment of robotic surgery

SUPR-GAN: SUrgical PRediction GAN for Event Anticipation in Laparoscopic and Robotic Surgery

Comprehension of surgical workflow is the foundation upon which artificial intelligence (AI) and machine learning (ML) holds the potential to assist intraoperative decision-making and risk mitigation. In this work, we move beyond mere identification of past surgical phases, into the prediction of future surgical steps and specification of the transitions between them. We use a novel Generative Adversarial Network (GAN) formulation to sample future surgical phases trajectories conditioned on past video frames from laparoscopic cholecystectomy (LC) videos and compare it to state-of-the-art approaches for surgical video analysis and alternative prediction methods. We demonstrate the GAN formulation's effectiveness through inferring and predicting the progress of LC videos. We quantify the horizon-accuracy trade-off and explored average performance, as well as the performance on the more challenging, and clinically relevant transitions between phases. Furthermore, we conduct a survey, asking 16 surgeons of different specialties and educational levels to qualitatively evaluate predicted surgery phases.Comment: RA-L ICRA 202

Postesophagectomy Diaphragmatic Prolapse after Robot-Assisted Minimally Invasive Esophagectomy (RAMIE)

Background: Postesophagectomy diaphragmatic prolapse (PDP) is a major complication after esophagectomy with significant mortality and morbidity. However, in the current literature, treatment and outcomes are not evaluated for patients undergoing an Ivor Lewis Robot-assisted minimally invasive esophagectomy (IL-RAMIE). The aim of this study is to evaluate the incidence of PDP after IL-RAMIE. Moreover, the study aims to determine whether using a minimally invasive approach in the management of PDP after an IL-RAMIE procedure is safe and feasible. Materials and Methods: This study includes all patients who received an IL-RAMIE at our high-volume center (>200 esophagectomies/year) between April 2017 and December 2022 and developed PDP. The analysis focuses on time to prolapse, symptoms, treatment, surgical method, and recurrence rates of these patients. Results: A total of 185 patients underwent an IL-RAMIE at our hospital. Eleven patients (5.9%) developed PDP. Patients presented with PDP after a medium time of 241 days with symptoms like reflux, nausea, vomiting, and pain. One-third of these patients did not suffer from any symptoms. In all cases, a CT scan was performed in which the colon transversum always presented as the herniated organ. In one patient, prolapse of the small intestine, pancreas, and greater omentum also occurred. A total of 91% of these patients received a revisional surgery in a minimally invasive manner with a mean hospital stay of 12 days. In four patients, PDP recurred (36%) after 13, 114, 119 and 237 days, respectively. Conclusion: This study shows that a minimally invasive approach in repositioning PDP is a safe and effective option after IL-RAMIE

Mapping the Lymphatic Drainage Pattern of Esophageal Cancer with Near-Infrared Fluorescent Imaging during Robotic Assisted Minimally Invasive Ivor Lewis Esophagectomy (RAMIE)—First Results of the Prospective ESOMAP Feasibility Trial

While the sentinel lymph node concept is routinely applied in other surgical fields, no established and valid modality for lymph node mapping for esophageal cancer surgery currently exists. Near-infrared light fluorescence (NIR) using indocyanine green (ICG) has been recently proven to be a safe technology for peritumoral injection and consecutive lymph node mapping in small surgical cohorts, mostly without the usage of robotic technology. The aim of this study was to identify the lymphatic drainage pattern of esophageal cancer during highly standardized RAMIE and to correlate the intraoperative images with the histopathological dissemination of lymphatic metastases. Patients with clinically advanced stage squamous cell carcinoma or adenocarcinoma of the esophagus undergoing a RAMIE at our Center of Excellence for Surgery of the Upper Gastrointestinal Tract were prospectively included in this study. Patients were admitted on the day prior to surgery, and an additional EGD with endoscopic injection of the ICG solution around the tumor was performed. Intraoperative imaging procedures were performed using the Stryker 1688 or the FIREFLY fluorescence imaging system, and resected lymph nodes were sent to pathology. A total of 20 patients were included in the study, and feasibility and safety for the application of NIR using ICG during RAMIE were shown. NIR imaging to detect lymph node metastases can be safely performed during RAMIE. Further analyses in our center will focus on pathological analyses of ICG-positive tissue and quantification using artificial intelligence tools with a correlation of long-term follow-up data

SAGES consensus recommendations on surgical video data use, structure, and exploration (for research in artificial intelligence, clinical quality improvement, and surgical education)

Abstract Background Surgery generates a vast amount of data from each procedure. Particularly video data provides significant value for surgical research, clinical outcome assessment, quality control, and education. The data lifecycle is influenced by various factors, including data structure, acquisition, storage, and sharing; data use and exploration, and finally data governance, which encompasses all ethical and legal regulations associated with the data. There is a universal need among stakeholders in surgical data science to establish standardized frameworks that address all aspects of this lifecycle to ensure data quality and purpose. Methods Working groups were formed, among 48 representatives from academia and industry, including clinicians, computer scientists and industry representatives. These working groups focused on: Data Use, Data Structure, Data Exploration, and Data Governance. After working group and panel discussions, a modified Delphi process was conducted. Results The resulting Delphi consensus provides conceptualized and structured recommendations for each domain related to surgical video data. We identified the key stakeholders within the data lifecycle and formulated comprehensive, easily understandable, and widely applicable guidelines for data utilization. Standardization of data structure should encompass format and quality, data sources, documentation, metadata, and account for biases within the data. To foster scientific data exploration, datasets should reflect diversity and remain adaptable to future applications. Data governance must be transparent to all stakeholders, addressing legal and ethical considerations surrounding the data. Conclusion This consensus presents essential recommendations around the generation of standardized and diverse surgical video databanks, accounting for multiple stakeholders involved in data generation and use throughout its lifecycle. Following the SAGES annotation framework, we lay the foundation for standardization of data use, structure, and exploration. A detailed exploration of requirements for adequate data governance will follow

Multicentric exploration of tool annotation in robotic surgery : lessons learned when starting a surgical artificial intelligence project

Background Artificial intelligence (AI) holds tremendous potential to reduce surgical risks and improve surgical assessment. Machine learning, a subfield of AI, can be used to analyze surgical video and imaging data. Manual annotations provide veracity about the desired target features. Yet, methodological annotation explorations are limited to date. Here, we provide an exploratory analysis of the requirements and methods of instrument annotation in a multi-institutional team from two specialized AI centers and compile our lessons learned. Methods We developed a bottom-up approach for team annotation of robotic instruments in robot-assisted partial nephrectomy (RAPN), which was subsequently validated in robot-assisted minimally invasive esophagectomy (RAMIE). Furthermore, instrument annotation methods were evaluated for their use in Machine Learning algorithms. Overall, we evaluated the efficiency and transferability of the proposed team approach and quantified performance metrics (e.g., time per frame required for each annotation modality) between RAPN and RAMIE. Results We found a 0.05 Hz image sampling frequency to be adequate for instrument annotation. The bottom-up approach in annotation training and management resulted in accurate annotations and demonstrated efficiency in annotating large datasets. The proposed annotation methodology was transferrable between both RAPN and RAMIE. The average annotation time for RAPN pixel annotation ranged from 4.49 to 12.6 min per image; for vector annotation, we denote 2.92 min per image. Similar annotation times were found for RAMIE. Lastly, we elaborate on common pitfalls encountered throughout the annotation process. Conclusions We propose a successful bottom-up approach for annotator team composition, applicable to any surgical annotation project. Our results set the foundation to start AI projects for instrument detection, segmentation, and pose estimation. Due to the immense annotation burden resulting from spatial instrumental annotation, further analysis into sampling frequency and annotation detail needs to be conducted

SAGES consensus recommendations on surgical video data use, structure, and exploration (for research in artificial intelligence, clinical quality improvement, and surgical education)

International audienceBackgroundSurgery generates a vast amount of data from each procedure. Particularly video data provides significant value for surgical research, clinical outcome assessment, quality control, and education. The data lifecycle is influenced by various factors, including data structure, acquisition, storage, and sharing; data use and exploration, and finally data governance, which encompasses all ethical and legal regulations associated with the data. There is a universal need among stakeholders in surgical data science to establish standardized frameworks that address all aspects of this lifecycle to ensure data quality and purpose.MethodsWorking groups were formed, among 48 representatives from academia and industry, including clinicians, computer scientists and industry representatives. These working groups focused on: Data Use, Data Structure, Data Exploration, and Data Governance. After working group and panel discussions, a modified Delphi process was conducted.ResultsThe resulting Delphi consensus provides conceptualized and structured recommendations for each domain related to surgical video data. We identified the key stakeholders within the data lifecycle and formulated comprehensive, easily understandable, and widely applicable guidelines for data utilization. Standardization of data structure should encompass format and quality, data sources, documentation, metadata, and account for biases within the data. To foster scientific data exploration, datasets should reflect diversity and remain adaptable to future applications. Data governance must be transparent to all stakeholders, addressing legal and ethical considerations surrounding the data.ConclusionThis consensus presents essential recommendations around the generation of standardized and diverse surgical video databanks, accounting for multiple stakeholders involved in data generation and use throughout its lifecycle. Following the SAGES annotation framework, we lay the foundation for standardization of data use, structure, and exploration. A detailed exploration of requirements for adequate data governance will follow