Fan-Slicer: A Pycuda Package for Fast Reslicing of Ultrasound Shaped Planes

Fan-Slicer (https://github.com/UCL/fan-slicer) is a Python package that enables the fast sampling (slicing) of 2D ultrasound-shaped images from a 3D volume. To increase sampling speed, CUDA kernel functions are used in conjunction with the Pycuda package. The main features include functions to generate images from both 3D surface models and 3D volumes. Additionally, the package also allows for the sampling of images from curvilinear (fan shaped planes) and linear (rectangle shaped planes) ultrasound transducers. Potential uses of Fan-slicer include the generation of large datasets of 2D images from 3D volumes and the simulation of intra-operative data among others

Registration of Untracked 2D Laparoscopic Ultrasound to CT Images of the Liver using Multi-Labelled Content-Based Image Retrieval

Laparoscopic Ultrasound (LUS) is recommended as a standard-of-care when performing laparoscopic liver resections as it images sub-surface structures such as tumours and major vessels. Given that LUS probes are difficult to handle and some tumours are iso-echoic, registration of LUS images to a pre-operative CT has been proposed as an image-guidance method. This registration problem is particularly challenging due to the small field of view of LUS, and usually depends on both a manual initialisation and tracking to compose a volume, hindering clinical translation. In this paper, we extend a proposed registration approach using Content-Based Image Retrieval (CBIR), removing the requirement for tracking or manual initialisation. Pre-operatively, a set of possible LUS planes is simulated from CT and a descriptor generated for each image. Then, a Bayesian framework is employed to estimate the most likely sequence of CT simulations that matches a series of LUS images. We extend our CBIR formulation to use multiple labelled objects and constrain the registration by separating liver vessels into portal vein and hepatic vein branches. The value of this new labeled approach is demonstrated in retrospective data from 5 patients. Results show that, by including a series of 5 untracked images in time, a single LUS image can be registered with accuracies ranging from 5.7 to 16.4 mm with a success rate of 78%. Initialisation of the LUS to CT registration with the proposed framework could potentially enable the clinical translation of these image fusion techniques

Vessel segmentation for automatic registration of untracked laparoscopic ultrasound to CT of the liver

PURPOSE: Registration of Laparoscopic Ultrasound (LUS) to a pre-operative scan such as Computed Tomography (CT) using blood vessel information has been proposed as a method to enable image-guidance for laparoscopic liver resection. Currently, there are solutions for this problem that can potentially enable clinical translation by bypassing the need for a manual initialisation and tracking information. However, no reliable framework for the segmentation of vessels in 2D untracked LUS images has been presented. METHODS: We propose the use of 2D UNet for the segmentation of liver vessels in 2D LUS images. We integrate these results in a previously developed registration method, and show the feasibility of a fully automatic initialisation to the LUS to CT registration problem without a tracking device. RESULTS: We validate our segmentation using LUS data from 6 patients. We test multiple models by placing patient datasets into different combinations of training, testing and hold-out, and obtain mean Dice scores ranging from 0.543 to 0.706. Using these segmentations, we obtain registration accuracies between 6.3 and 16.6 mm in 50% of cases. CONCLUSIONS: We demonstrate the first instance of deep learning (DL) for the segmentation of liver vessels in LUS. Our results show the feasibility of UNet in detecting multiple vessel instances in 2D LUS images, and potentially automating a LUS to CT registration pipeline

Global rigid registration of CT to video in laparoscopic liver surgery

PURPOSE: Image-guidance systems have the potential to aid in laparoscopic interventions by providing sub-surface structure information and tumour localisation. The registration of a preoperative 3D image with the intraoperative laparoscopic video feed is an important component of image guidance, which should be fast, robust and cause minimal disruption to the surgical procedure. Most methods for rigid and non-rigid registration require a good initial alignment. However, in most research systems for abdominal surgery, the user has to manually rotate and translate the models, which is usually difficult to perform quickly and intuitively. METHODS: We propose a fast, global method for the initial rigid alignment between a 3D mesh derived from a preoperative CT of the liver and a surface reconstruction of the intraoperative scene. We formulate the shape matching problem as a quadratic assignment problem which minimises the dissimilarity between feature descriptors while enforcing geometrical consistency between all the feature points. We incorporate a novel constraint based on the liver contours which deals specifically with the challenges introduced by laparoscopic data. RESULTS: We validate our proposed method on synthetic data, on a liver phantom and on retrospective clinical data acquired during a laparoscopic liver resection. We show robustness over reduced partial size and increasing levels of deformation. Our results on the phantom and on the real data show good initial alignment, which can successfully converge to the correct position using fine alignment techniques. Furthermore, since we can pre-process the CT scan before surgery, the proposed method runs faster than current algorithms. CONCLUSION: The proposed shape matching method can provide a fast, global initial registration, which can be further refined by fine alignment methods. This approach will lead to a more usable and intuitive image-guidance system for laparoscopic liver surgery

SnappySonic: An Ultrasound Acquisition Replay Simulator

SnappySonic provides an ultrasound acquisition replay simulator designed for public engagement and training. It provides a simple interface to allow users to experience ultrasound acquisition without the need for specialist hardware or acoustically compatible phantoms. The software is implemented in Python, built on top of a set of open source Python modules targeted at surgical innovation. The library has high potential for reuse, most obviously for those who want to simulate ultrasound acquisition, but it could also be used as a user interface for displaying high dimensional images or video data

Intelligent viewpoint selection for efficient CT to video registration in laparoscopic liver surgery

PURPOSE: Minimally invasive surgery offers advantages over open surgery due to a shorter recovery time, less pain and trauma for the patient. However, inherent challenges such as lack of tactile feedback and difficulty in controlling bleeding lower the percentage of suitable cases. Augmented reality can show a better visualisation of sub-surface structures and tumour locations by fusing pre-operative CT data with real-time laparoscopic video. Such augmented reality visualisation requires a fast and robust video to CT registration that minimises interruption to the surgical procedure. METHODS: We propose to use view planning for efficient rigid registration. Given the trocar position, a set of camera positions are sampled and scored based on the corresponding liver surface properties. We implement a simulation framework to validate the proof of concept using a segmented CT model from a human patient. Furthermore, we apply the proposed method on clinical data acquired during a human liver resection. RESULTS: The first experiment motivates the viewpoint scoring strategy and investigates reliable liver regions for accurate registrations in an intuitive visualisation. The second experiment shows wider basins of convergence for higher scoring viewpoints. The third experiment shows that a comparable registration performance can be achieved by at least two merged high scoring views and four low scoring views. Hence, the focus could change from the acquisition of a large liver surface to a small number of distinctive patches, thereby giving a more explicit protocol for surface reconstruction. We discuss the application of the proposed method on clinical data and show initial results. CONCLUSION: The proposed simulation framework shows promising results to motivate more research into a comprehensive view planning method for efficient registration in laparoscopic liver surgery

Image-guidance in endoscopic pituitary surgery: an in-silico study of errors involved in tracker-based techniques

Background: Endoscopic endonasal surgery is an established minimally invasive technique for resecting pituitary adenomas. However, understanding orientation and identifying critical neurovascular structures in this anatomically dense region can be challenging. In clinical practice, commercial navigation systems use a tracked pointer for guidance. Augmented Reality (AR) is an emerging technology used for surgical guidance. It can be tracker based or vision based, but neither is widely used in pituitary surgery. Methods: This pre-clinical study aims to assess the accuracy of tracker-based navigation systems, including those that allow for AR. Two setups were used to conduct simulations: (1) the standard pointer setup, tracked by an infrared camera; and (2) the endoscope setup that allows for AR, using reflective markers on the end of the endoscope, tracked by infrared cameras. The error sources were estimated by calculating the Euclidean distance between a point’s true location and the point’s location after passing it through the noisy system. A phantom study was then conducted to verify the in-silico simulation results and show a working example of image-based navigation errors in current methodologies. Results: The errors of the tracked pointer and tracked endoscope simulations were 1.7 and 2.5 mm respectively. The phantom study showed errors of 2.14 and 3.21 mm for the tracked pointer and tracked endoscope setups respectively. Discussion: In pituitary surgery, precise neighboring structure identification is crucial for success. However, our simulations reveal that the errors of tracked approaches were too large to meet the fine error margins required for pituitary surgery. In order to achieve the required accuracy, we would need much more accurate tracking, better calibration and improved registration techniques

Registration of Untracked 2D Laparoscopic Ultrasound Liver Images to CT Using Content-Based Retrieval and Kinematic Priors

Laparoscopic Ultrasound (LUS) can enhance the safety of laparoscopic liver resection by providing information on the location of major blood vessels and tumours. Since many tumours are not visible in ultrasound, registration to a pre-operative CT has been proposed as a guidance method. In addition to being multi-modal, this registration problem is greatly affected by the differences in field of view between CT and LUS, and thus requires an accurate initialisation. We propose a novel method of registering smaller field of view slices to a larger volume globally using a Content-based retrieval framework. This problem is under-constrained for a single slice registration, resulting in non-unique solutions. Therefore, we introduce kinematic priors in a Bayesian framework in order to jointly register groups of ultrasound images. Our method then produces an estimate of the most likely sequence of CT images to represent the ultrasound acquisition and does not require tracking information nor an accurate initialisation. We demonstrate the feasibility of this approach in multiple LUS acquisitions taken from three sets of clinical data

On Pattern Selection for Laparoscope Calibration

Camera calibration is a key requirement for augmented reality in surgery. Calibration of laparoscopes provides two challenges that are not sufficiently addressed in the literature. In the case of stereo laparoscopes the small distance (less than 5mm) between the channels means that the calibration pattern is an order of magnitude more distant than the stereo separation. For laparoscopes in general, if an external tracking system is used, hand-eye calibration is difficult due to the long length of the laparoscope. Laparoscope intrinsic, stereo and hand-eye calibration all rely on accurate feature point selection and accurate estimation of the camera pose with respect to a calibration pattern. We compare 3 calibration patterns, chessboard, rings, and AprilTags. We measure the error in estimating the camera intrinsic parameters and the camera poses. Accuracy of camera pose estimation will determine the accuracy with which subsequent stereo or hand-eye calibration can be done. We compare the results of repeated real calibrations and simulations using idealised noise, to determine the expected accuracy of different methods and the sources of error. The results do indicate that feature detection based on rings is more accurate than a chessboard, however this doesn’t necessarily lead to a better calibration. Using a grid with identifiable tags enables detection of features nearer the image boundary, which may improve calibration