Towards a Video Quality Assessment based Framework for Enhancement of Laparoscopic Videos

Laparoscopic videos can be affected by different distortions which may impact the performance of surgery and introduce surgical errors. In this work, we propose a framework for automatically detecting and identifying such distortions and their severity using video quality assessment. There are three major contributions presented in this work (i) a proposal for a novel video enhancement framework for laparoscopic surgery; (ii) a publicly available database for quality assessment of laparoscopic videos evaluated by expert as well as non-expert observers and (iii) objective video quality assessment of laparoscopic videos including their correlations with expert and non-expert scores.Comment: SPIE Medical Imaging 2020 (Draft version

Influence of sampling accuracy on augmented reality for laparoscopic image-guided surgery

Purpose This study aims to evaluate the accuracy of point-based registration (PBR) when used for augmented reality (AR) in laparoscopic liver resection surgery. Material and methods The study was conducted in three different scenarios in which the accuracy of sampling targets for PBR decreases: using an assessment phantom with machined divot holes, a patient-specific liver phantom with markers visible in computed tomography (CT) scans and in vivo, relying on the surgeon’s anatomical understanding to perform annotations. Target registration error (TRE) and fiducial registration error (FRE) were computed using five randomly selected positions for image-to-patient registration. Results AR with intra-operative CT scanning showed a mean TRE of 6.9 mm for the machined phantom, 7.9 mm for the patient-specific phantom and 13.4 mm in the in vivo study. Conclusions AR showed an increase in both TRE and FRE throughout the experimental studies, proving that AR is not robust to the sampling accuracy of the targets used to compute image-to-patient registration. Moreover, an influence of the size of the volume to be register was observed. Hence, it is advisable to reduce both errors due to annotations and the size of registration volumes, which can cause large errors in AR systems

Navigated liver surgery: State of the art and future perspectives.

BACKGROUND In recent years, the development of digital imaging technology has had a significant influence in liver surgery. The ability to obtain a 3-dimensional (3D) visualization of the liver anatomy has provided surgery with virtual reality of simulation 3D computer models, 3D printing models and more recently holograms and augmented reality (when virtual reality knowledge is superimposed onto reality). In addition, the utilization of real-time fluorescent imaging techniques based on indocyanine green (ICG) uptake allows clinicians to precisely delineate the liver anatomy and/or tumors within the parenchyma, applying the knowledge obtained preoperatively through digital imaging. The combination of both has transformed the abstract thinking until now based on 2D imaging into a 3D preoperative conception (virtual reality), enhanced with real-time visualization of the fluorescent liver structures, effectively facilitating intraoperative navigated liver surgery (augmented reality). DATA SOURCES A literature search was performed from inception until January 2021 in MEDLINE (PubMed), Embase, Cochrane library and database for systematic reviews (CDSR), Google Scholar, and National Institute for Health and Clinical Excellence (NICE) databases. RESULTS Fifty-one pertinent articles were retrieved and included. The different types of digital imaging technologies and the real-time navigated liver surgery were estimated and compared. CONCLUSIONS ICG fluorescent imaging techniques can contribute essentially to the real-time definition of liver segments; as a result, precise hepatic resection can be guided by the presence of fluorescence. Furthermore, 3D models can help essentially to further advancing of precision in hepatic surgery by permitting estimation of liver volume and functional liver remnant, delineation of resection lines along the liver segments and evaluation of tumor margins. In liver transplantation and especially in living donor liver transplantation (LDLT), 3D printed models of the donor's liver and models of the recipient's hilar anatomy can contribute further to improving the results. In particular, pediatric LDLT abdominal cavity models can help to manage the largest challenge of this procedure, namely large-for-size syndrome

Laparoscopic parenchyma-sparing liver resection for large (≥ 50 mm) colorectal metastases.

Abstract Background Traditionally, patients with large liver tumors (≥ 50 mm) have been considered for anatomic major hepatectomy. Laparoscopic resection of large liver lesions is technically challenging and often performed by surgeons with extensive experience. The current study aimed to evaluate the surgical and oncologic safety of laparoscopic parenchyma-sparing liver resection in patients with large colorectal metastases. Methods Patients who primarily underwent laparoscopic parenchyma-sparing liver resection (less than 3 consecutive liver segments) for colorectal liver metastases between 1999 and 2019 at Oslo University Hospital were analyzed. In some recent cases, a computer-assisted surgical planning system was used to better visualize and understand the patients’ liver anatomy, as well as a tool to further improve the resection strategy. The surgical and oncologic outcomes of patients with large (≥ 50 mm) and small (< 50 mm) tumors were compared. Multivariable Cox-regression analysis was performed to identify risk factors for survival. Results In total 587 patients met the inclusion criteria (large tumor group, n  = 59; and small tumor group, n  = 528). Median tumor size was 60 mm (range, 50–110) in the large tumor group and 21 mm (3–48) in the small tumor group ( p  < 0.001). Patient age and CEA level were higher in the large tumor group (8.4 μg/L vs. 4.6 μg/L, p  < 0.001). Operation time and conversion rate were similar, while median blood loss was higher in the large tumor group (500 ml vs. 200 ml, p  < 0.001). Patients in the large tumor group had shorter 5 year overall survival (34% vs 49%, p  = 0.027). However, in the multivariable Cox-regression analysis tumor size did not impact survival, unlike parameters such as age, ASA score, CEA level, extrahepatic disease at liver surgery, and positive lymph nodes in the primary tumor. Conclusion Laparoscopic parenchyma-sparing resections for large colorectal liver metastases provide satisfactory short and long-term outcomes. Graphical abstrac

Learning deep abdominal CT registration through adaptive loss weighting and synthetic data generation.

PurposeThis study aims to explore training strategies to improve convolutional neural network-based image-to-image deformable registration for abdominal imaging.MethodsDifferent training strategies, loss functions, and transfer learning schemes were considered. Furthermore, an augmentation layer which generates artificial training image pairs on-the-fly was proposed, in addition to a loss layer that enables dynamic loss weighting.ResultsGuiding registration using segmentations in the training step proved beneficial for deep-learning-based image registration. Finetuning the pretrained model from the brain MRI dataset to the abdominal CT dataset further improved performance on the latter application, removing the need for a large dataset to yield satisfactory performance. Dynamic loss weighting also marginally improved performance, all without impacting inference runtime.ConclusionUsing simple concepts, we improved the performance of a commonly used deep image registration architecture, VoxelMorph. In future work, our framework, DDMR, should be validated on different datasets to further assess its value

Use of mixed reality for improved spatial understanding of liver anatomy

Introduction: In liver surgery, medical images from pre-operative computed tomography and magnetic resonance imaging are the basis for the decision-making process. These images are used in surgery planning and guidance, especially for parenchyma-sparing hepatectomies. Though medical images are commonly visualized in two dimensions (2D), surgeons need to mentally reconstruct this information in three dimensions (3D) for a spatial understanding of the anatomy. The aim of this work is to investigate whether the use of a 3D model visualized in mixed reality with Microsoft HoloLens increases the spatial understanding of the liver, compared to the conventional way of using 2D images. Material and methods: In this study, clinicians had to identify liver segments associated to lesions. Results: Twenty-eight clinicians with varying medical experience were recruited for the study. From a total of 150 lesions, 89 were correctly assigned without significant difference between the modalities. The median time for correct identification was 23.5 [4–138] s using the magnetic resonance imaging images and 6.00 [1–35] s using HoloLens (p < 0.001). Conclusions: The use of 3D liver models in mixed reality significantly decreases the time for tasks requiring a spatial understanding of the organ. This may significantly decrease operating time and improve use of resources

The effect of intraoperative imaging on surgical navigation for laparoscopic liver resection surgery

Conventional surgical navigation systems rely on preoperative imaging to provide guidance. In laparoscopic liver surgery, insufflation of the abdomen (pneumoperitoneum) can cause deformations on the liver, introducing inaccuracies in the correspondence between the preoperative images and the intraoperative reality. This study evaluates the improvements provided by intraoperative imaging for laparoscopic liver surgical navigation, when displayed as augmented reality (AR). Significant differences were found in terms of accuracy of the AR, in favor of intraoperative imaging. In addition, results showed an effect of user-induced error: image-to-patient registration based on annotations performed by clinicians caused 33% more inaccuracy as compared to image-to-patient registration algorithms that do not depend on user annotations. Hence, to achieve accurate surgical navigation for laparoscopic liver surgery, intraoperative imaging is recommendable to compensate for deformation. Moreover, user annotation errors may lead to inaccuracies in registration processes

The effect of intraoperative imaging on surgical navigation for laparoscopic liver resection surgery

Conventional surgical navigation systems rely on preoperative imaging to provide guidance. In laparoscopic liver surgery, insufflation of the abdomen (pneumoperitoneum) can cause deformations on the liver, introducing inaccuracies in the correspondence between the preoperative images and the intraoperative reality. This study evaluates the improvements provided by intraoperative imaging for laparoscopic liver surgical navigation, when displayed as augmented reality (AR). Significant differences were found in terms of accuracy of the AR, in favor of intraoperative imaging. In addition, results showed an effect of user-induced error: image-to-patient registration based on annotations performed by clinicians caused 33% more inaccuracy as compared to image-to-patient registration algorithms that do not depend on user annotations. Hence, to achieve accurate surgical navigation for laparoscopic liver surgery, intraoperative imaging is recommendable to compensate for deformation. Moreover, user annotation errors may lead to inaccuracies in registration processes

Use of mixed reality for improved spatial understanding of liver anatomy

Introduction: In liver surgery, medical images from pre-operative computed tomography and magnetic resonance imaging are the basis for the decision-making process. These images are used in surgery planning and guidance, especially for parenchyma-sparing hepatectomies. Though medical images are commonly visualized in two dimensions (2D), surgeons need to mentally reconstruct this information in three dimensions (3D) for a spatial understanding of the anatomy. The aim of this work is to investigate whether the use of a 3D model visualized in mixed reality with Microsoft HoloLens increases the spatial understanding of the liver, compared to the conventional way of using 2D images. Material and methods: In this study, clinicians had to identify liver segments associated to lesions. Results: Twenty-eight clinicians with varying medical experience were recruited for the study. From a total of 150 lesions, 89 were correctly assigned without significant difference between the modalities. The median time for correct identification was 23.5 [4–138] s using the magnetic resonance imaging images and 6.00 [1–35] s using HoloLens (p < 0.001). Conclusions: The use of 3D liver models in mixed reality significantly decreases the time for tasks requiring a spatial understanding of the organ. This may significantly decrease operating time and improve use of resources