205 research outputs found
Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery
One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions
Automatic registration of 3D models to laparoscopic video images for guidance during liver surgery
Laparoscopic liver interventions offer significant advantages over open surgery, such as less pain and trauma, and shorter recovery time for the patient. However, they also bring challenges for the surgeons such as the lack of tactile feedback, limited field of view and occluded anatomy. Augmented reality (AR) can potentially help during laparoscopic liver interventions by displaying sub-surface structures (such as tumours or vasculature). The initial registration between the 3D model extracted from the CT scan and the laparoscopic video feed is essential for an AR system which should be efficient, robust, intuitive to use and with minimal disruption to the surgical procedure. Several challenges of registration methods in laparoscopic interventions include the deformation of the liver due to gas insufflation in the abdomen, partial visibility of the organ and lack of prominent geometrical or texture-wise landmarks. These challenges are discussed in detail and an overview of the state of the art is provided. This research project aims to provide the tools to move towards a completely automatic registration. Firstly, the importance of pre-operative planning is discussed along with the characteristics of the liver that can be used in order to constrain a registration method. Secondly, maximising the amount of information obtained before the surgery, a semi-automatic surface based method is proposed to recover the initial rigid registration irrespective of the position of the shapes. Finally, a fully automatic 3D-2D rigid global registration is proposed which estimates a global alignment of the pre-operative 3D model using a single intra-operative image. Moving towards incorporating the different liver contours can help constrain the registration, especially for partial surfaces. Having a robust, efficient AR system which requires no manual interaction from the surgeon will aid in the translation of such approaches to the clinics
Global rigid registration of CT to video in laparoscopic liver surgery
PURPOSE: Image-guidance systems have the potential to aid in laparoscopic interventions by providing sub-surface structure information and tumour localisation. The registration of a preoperative 3D image with the intraoperative laparoscopic video feed is an important component of image guidance, which should be fast, robust and cause minimal disruption to the surgical procedure. Most methods for rigid and non-rigid registration require a good initial alignment. However, in most research systems for abdominal surgery, the user has to manually rotate and translate the models, which is usually difficult to perform quickly and intuitively. METHODS: We propose a fast, global method for the initial rigid alignment between a 3D mesh derived from a preoperative CT of the liver and a surface reconstruction of the intraoperative scene. We formulate the shape matching problem as a quadratic assignment problem which minimises the dissimilarity between feature descriptors while enforcing geometrical consistency between all the feature points. We incorporate a novel constraint based on the liver contours which deals specifically with the challenges introduced by laparoscopic data. RESULTS: We validate our proposed method on synthetic data, on a liver phantom and on retrospective clinical data acquired during a laparoscopic liver resection. We show robustness over reduced partial size and increasing levels of deformation. Our results on the phantom and on the real data show good initial alignment, which can successfully converge to the correct position using fine alignment techniques. Furthermore, since we can pre-process the CT scan before surgery, the proposed method runs faster than current algorithms. CONCLUSION: The proposed shape matching method can provide a fast, global initial registration, which can be further refined by fine alignment methods. This approach will lead to a more usable and intuitive image-guidance system for laparoscopic liver surgery
Learning Feature Descriptors for Pre- and Intra-operative Point Cloud Matching for Laparoscopic Liver Registration
Purpose: In laparoscopic liver surgery (LLS), pre-operative information can
be overlaid onto the intra-operative scene by registering a 3D pre-operative
model to the intra-operative partial surface reconstructed from the
laparoscopic video. To assist with this task, we explore the use of
learning-based feature descriptors, which, to our best knowledge, have not been
explored for use in laparoscopic liver registration. Furthermore, a dataset to
train and evaluate the use of learning-based descriptors does not exist.
Methods: We present the LiverMatch dataset consisting of 16 preoperative
models and their simulated intra-operative 3D surfaces. We also propose the
LiverMatch network designed for this task, which outputs per-point feature
descriptors, visibility scores, and matched points.
Results: We compare the proposed LiverMatch network with anetwork closest to
LiverMatch, and a histogram-based 3D descriptor on the testing split of the
LiverMatch dataset, which includes two unseen pre-operative models and 1400
intra-operative surfaces. Results suggest that our LiverMatch network can
predict more accurate and dense matches than the other two methods and can be
seamlessly integrated with a RANSAC-ICP-based registration algorithm to achieve
an accurate initial alignment.
Conclusion: The use of learning-based feature descriptors in LLR is
promising, as it can help achieve an accurate initial rigid alignment, which,
in turn, serves as an initialization for subsequent non-rigid registration. We
will release the dataset and code upon acceptance
Image-guided Simulation of Heterogeneous Tissue Deformation For Augmented Reality during Hepatic Surgery
International audienceThis paper presents a method for real-time augmentation of vas- cular network and tumors during minimally invasive liver surgery. Internal structures computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Com- pared to state-of-the-art methods, our method uses a real-time biomechanical model to compute a volumetric displacement field from partial three-dimensional liver surface motion. This permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Real-time augmentation results are presented on in vivo and ex vivo data and illustrate the benefits of such an approach for minimally invasive surgery
Intraoperative Biomechanical Registration of the Liver: Does the Heterogeneity of the Liver Matter?
International audienceBackground: Preoperative images such as computed tomography scans or magnetic resonance imaging contain lots of valuable information that are not easily available for surgeons during an operation. To help the clinicians better target the structures of interest during an intervention, many registration methods that align preoperative images onto the intraoperative view of the organs have been developed. For important organ deformation, biomechanically-based registration has proven to be a method of choice.Method: Using an existing biomechanically-based registration algorithm for laparoscopic liver surgery we investigate in this paper the influence of the heterogeneity of the liver on the registration result.Results: No statistical difference in the results was found between the registration performed with the homogeneous model and the one carried out with the heterogeneous model.Conclusion: As the use of an heterogeneous model does not improve significantly the registration result and increase the computation time we recommend to perform the type of registration task described in the paper with a simplified homogeneous model
Image-Aligned Dynamic Liver Reconstruction Using Intra-Operative Field of Views for Minimal Invasive Surgery
Available online on 30 November 2018. Author's post-print on open access repository after an embargo period of 12 months2019-11-3
Impact of Soft Tissue Heterogeneity on Augmented Reality for Liver Surgery
International audienceThis paper presents a method for real-time augmented reality of internal liver structures during minimally invasive hepatic surgery. Vessels and tumors computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Compared to current methods, our method is able to locate the in-depth positions of the tumors based on partial three-dimensional liver tissue motion using a real-time biomechanical model. This model permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Experimentations conducted on phantom liver permits to measure the accuracy of the augmentation while real-time augmentation on in vivo human liver during real surgery shows the benefits of such an approach for minimally invasive surgery
- …