A reference-based approach for tumor size estimation in monocular laparoscopic videos

Laparoscopic exploration of the abdominal cavity is rou tinely performed for the diagnosis, assessment, and staging of peritoneal metastasis (PM). Accurately measuring tumor size during this procedure is crucial for prognosis and treatment planning. As conventional approaches for tumor size measurement rely on subjective manual assessments during or after surgery, they stand to benefit from computer assistance. This study proposes a new method for measuring tumor size in laparoscopic monocular videos. Specifically, we introduce a novel mathematical equation that connects the intrinsic parameters of a monocular camera, the surface area of target and reference objects, and their distances to the camera. Furthermore, we combine this equation with an object segmentation model (Mask2Former) and a depth estimation model (MiDaS), creating an end-to-end framework that automates tumor size measurement in monocular laparoscopic videos. We evaluate the proposed method using a laparoscopy dataset comprising 18 videos depicting 76 tumor biopsies, with tumor size measured by surgeons who are experts in laparoscopic surgery. When estimating the size of the various tumors in this dataset, we obtain a Mean Absolute Error (MAE) of 2.44 mm ± 0.23 mm, demonstrating that the newly proposed method accurately predicts intraoperative tumor size. Our code and the evaluation dataset are publicly available on https://github.com/amiiiirrrr/TSEML

Development of a video-based deep learning model for differentiation of malignant and benign lesions during staging laparoscopy : is the machine better than the expert?

Background: Laparoscopic staging of abdominal cancer is routinely performed to assess the presence of peritoneal metastasis (PM). One major challenge of laparoscopic staging is the ability to distinguish between a malignant and benign peritoneal lesion, particularly scar tissue. Lesions' recognition is subjective to intra-case variability, surgeons' experience, type of primary tumor and response to systemic chemotherapy. The Peritoneal Regression Grading Score (PRGS) offers an objective histologic evaluation of the biopsied lesion by assessing the complete or partial presence of malignant cells (PRGS 2-3-4) and fibrous or benign lesion (PRGS 1). Machine learning-based (ML-based) computer vision has various applications in the medical field and can be useful in deducing clinical information through surgical video analysis. The aim of this study was to develop a ML model that can aid the surgeon in the intra-operative assessment of PM. Methods: Retrospectively collected videos of laparoscopies for the staging of PM before cytoreductive surgery or PIPAC were screened from the institutional database for the presence of recorded biopsies. PRGS from the pathology report of these biopsies was retrieved and reviewed by a pathologist. The surgical phase of the biopsy was annotated by systematic selection of 200 frames before the closure of the biopsy instrument on the tissue. One single frame was selected from each timeframe based on its representativeness. Two trained annotators performed the selection, one surgical expert reviewed each annotation. Discrepancies were solved by consensus. ML models based on multiple (ResEfficientnetV2_L) and single frames (Resnet50) per biopsy were trained on the PRGS using a one-versus-all binary classification. Five-fold cross-validation was applied. Two oncologic surgeons, experts in treating and assessing PM, blindly and independently scored the biopsies with PRGS. Results: A total of 127 videos from 67 patients were identified, showcasing PM of gastric, colorectal, appendix, hepatic, gallbladder, breast carcinomas, primary peritoneal tumors and benign lesions. Annotation was performed on 463 biopsies: PRGS 1: 204, PRGS 2: 164, PRGS 3: 70, PRGS 4: 25. Based on timeframe annotation 5214 (PRGS 1: 2044, PRGS 2: 1814, PRGS 3: 929, PRGS 4: 427) images were identified. The model trained on one-versus-all classification for single and multiple images showed an accuracy of 52.4% (Precision 56.2, Recall 48.2) and 45.7% (Precision 76.8, Recall 36.0) respectively in the validation set. Accuracy of each expert was 36.0% and 30.5%. Conclusions: While the accuracy of computer vision models may not be clinically acceptable as a substitute for biopsies, they outperform surgeons in differentiating PM from benign lesions. Computer vision models might be of aid for the surgeon in the recognition of these lesions