Segmentation, separation and pose estimation of prostate brachytherapy seeds in CT images.

International audienceIn this paper, we address the development of an automatic approach for the computation of pose information (position + orientation) of prostate brachytherapy loose seeds from 3D CT images. From an initial detection of a set of seed candidates in CT images using a threshold and connected component method, the orientation of each individual seed is estimated by using the principal components analysis (PCA) method. The main originality of this approach is the ability to classify the detected objects based on a priori intensity and volume information and to separate groups of closely spaced seeds using three competing clustering methods: the standard and a modified k-means method and a Gaussian mixture model with an Expectation-Maximization algorithm. Experiments were carried out on a series of CT images of two phantoms and patients. The fourteen patients correspond to a total of 1063 implanted seeds. Detections are compared to manual segmentation and to related work in terms of detection performance and calculation time. This automatic method has proved to be accurate and fast including the ability to separate groups of seeds in a reliable way and to determine the orientation of each seed. Such a method is mandatory to be able to compute precisely the real dose delivered to the patient post-operatively instead of assuming the alignment of seeds along the theoretical insertion direction of the brachytherapy needles

A Fast and Robust Image-Based Method for tracking Robot-assisted Needle Placement in Real-time MR Images

This thesis deals with automatic localization and tracking of surgical tools such as needles in Magnetic Resonance Imaging(MRI). The accurate and precise localization of needles is very important for medical interventions such as biopsy, brachytherapy, anaesthesia and many other needle based percutaneous interventions. Needle tracking has to be really precise, because the target may reside adjacent to organs which are sensitive to injury. More over during the needle insertion, Magnetic Resonance Imaging(MRI) scan plane must be aligned such that needle is in the field of view (FOV) for surgeon. Many approaches were proposed for needle tracking and automatic MRI scan plane control over last decade that use external markers, but they are not able to account for possible needle bending. Significant amount of work has already been done by using the image based approaches for needle tracking in Image Guided Therapy (IGT) but the existing approaches for surgical robots under MRI guidance are purely based on imaging information; they are missing the important fact that, a lot of important information (for example, depth of insertion, entry point and angle of insertion) is available from the kinematic model of the robot. The existing approaches are also not considering the fact that the needle insertion results in a time sequence of images. So the information about needle positions from the images seen so far can be used to make an approximate estimate about the needle position in the subsequent images. During the course of this thesis we have investigated an image based approach for needle tracking in real-time MR images that leverages additional information available from robot\u27s kinematics model, supplementing the acquired images. The proposed approach uses Standard Hough Transform(SHT) for needle detection in 2D MR image and uses Kalman Filter for tracking the needle over the sequence of images. We have demonstrated experimental validation of the method on Real MRI data using gel phantom and artificially created test images. The results proved that the proposed method can track the needle tip position with root mean squared error of 1.5 mm for straight needle and 2.5mm for curved needle

Tools for improving high-dose-rate prostate cancer brachytherapy using three-dimensional ultrasound and magnetic resonance imaging

High-dose-rate brachytherapy (HDR-BT) is an interstitial technique for the treatment of intermediate and high-risk localized prostate cancer that involves placement of a radiation source directly inside the prostate using needles. Dose-escalated whole-gland treatments have led to improvements in survival, and tumour-targeted treatments may offer future improvements in therapeutic ratio. The efficacy of tumour-targeted HDR-BT depends on imaging tools to enable accurate dose delivery to prostate sub-volumes. This thesis is focused on implementing ultrasound tools to improve HDR-BT needle localization accuracy and efficiency, and evaluating dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) for tumour localization. First, we implemented a device enabling sagittally-reconstructed 3D (SR3D) ultrasound, which provides sub-millimeter resolution in the needle insertion direction. We acquired SR3D and routine clinical images in a cohort of 12 consecutive eligible HDR-BT patients, with a total of 194 needles. The SR3D technique provided needle insertion depth errors within 5 mm for 93\% of needles versus 76\% for the clinical imaging technique, leading to increased precision in dose delivered to the prostate. Second, we implemented an algorithm to automatically segment multiple HDR-BT needles in a SR3D image. The algorithm was applied to the SR3D images from the first patient cohort, demonstrating mean execution times of 11.0 s per patient and successfully segmenting 82\% of needles within 3 mm. Third, we augmented SR3D imaging with live-2D sagittal ultrasound for needle tip localization. This combined technique was applied to another cohort of 10 HDR-BT patients, reducing insertion depth errors compared to routine imaging from a range of [-8.1 mm, 7.7 mm] to [-6.2 mm, 5.9 mm]. Finally, we acquired DCE-MRI in 16 patients scheduled to undergo prostatectomy, using either high spatial resolution or high temporal resolution imaging, and compared the images to whole-mount histology. The high spatial resolution images demonstrated improved high-grade cancer classification compared to the high temporal resolution images, with areas under the receiver operating characteristic curve of 0.79 and 0.70, respectively. In conclusion, we have translated and evaluated specialized imaging tools for HDR-BT which are ready to be tested in a clinical trial investigating tumour-targeted treatment

Transrectal ultrasound image processing for brachytherapy applications

In this thesis, we propose a novel algorithm for detecting needles and their corresponding implanted radioactive seed locations in the prostate. The seed localization process is carried out efficiently using separable Gaussian filters in a probabilistic Gibbs random field framework. An approximation of the needle path through the prostate volume is obtained using a polynomial fit. The seeds are then detected and assigned to their corresponding needles by calculating local maxima of the voronoi region around the needle position. In our experiments, we were able to successfully localize over 85% of the implanted seeds. Furthermore, as a regular part of a brachytherapy cancer treatment, patient’s prostate is scanned using a trans-rectal ultrasound probe, its boundary is manually outlined, and its volume is estimated for dosimetry purposes. In this thesis, we also propose a novel semi-automatic segmentation algorithm for prostate boundary detection that requires a reduced amount of radiologist’s input, and thus speeds up the surgical procedure. Saved time can be used to re-scan the prostate during the operation and accordingly adjust the treatment plan. The proposed segmentation algorithm utilizes texture differences between ultrasound images of the prostate tissue and the surrounding tissues. It is carried out in 5 the polar coordinate system and it uses three-dimensional data correlation to improve the smoothness and reliability of the segmentation. Test results show that the boundary segmentation obtained from the algorithm can reduce manual input by the factor of 3, without significantly affecting the accuracy of the segmentation (i.e. semi-automatically estimated prostate volume is within 90% of the original estimate)

Improving needle visibility in LED-based photoacoustic imaging using deep learning with semi-synthetic datasets

Photoacoustic imaging has shown great potential for guiding minimally invasive procedures by accurate identification of critical tissue targets and invasive medical devices (such as metallic needles). The use of light emitting diodes (LEDs) as the excitation light sources accelerates its clinical translation owing to its high affordability and portability. However, needle visibility in LED-based photoacoustic imaging is compromised primarily due to its low optical fluence. In this work, we propose a deep learning framework based on U-Net to improve the visibility of clinical metallic needles with a LED-based photoacoustic and ultrasound imaging system. To address the complexity of capturing ground truth for real data and the poor realism of purely simulated data, this framework included the generation of semi-synthetic training datasets combining both simulated data to represent features from the needles and in vivo measurements for tissue background. Evaluation of the trained neural network was performed with needle insertions into blood-vessel-mimicking phantoms, pork joint tissue ex vivo and measurements on human volunteers. This deep learning-based framework substantially improved the needle visibility in photoacoustic imaging in vivo compared to conventional reconstruction by suppressing background noise and image artefacts, achieving 5.8 and 4.5 times improvements in terms of signal-to-noise ratio and the modified Hausdorff distance, respectively. Thus, the proposed framework could be helpful for reducing complications during percutaneous needle insertions by accurate identification of clinical needles in photoacoustic imaging

Freehand 2D Ultrasound Probe Calibration for Image Fusion with 3D MRI/CT

The aim of this work is to implement a simple freehand ultrasound (US) probe calibration technique. This will enable us to visualize US image data during surgical procedures using augmented reality. The performance of the system was evaluated with different experiments using two different pose estimation techniques. A near-millimeter accuracy can be achieved with the proposed approach. The developed system is cost-effective, simple and rapid with low calibration erro

A hybrid camera- and ultrasound-based approach for needle localization and tracking using a 3D motorized curvilinear ultrasound probe

Three-dimensional (3D) motorized curvilinear ultrasound probes provide an effective, low-cost tool to guide needle interventions, but localizing and tracking the needle in 3D ultrasound volumes is often challenging. In this study, a new method is introduced to localize and track the needle using 3D motorized curvilinear ultrasound probes. In particular, a low-cost camera mounted on the probe is employed to estimate the needle axis. The camera-estimated axis is used to identify a volume of interest (VOI) in the ultrasound volume that enables high needle visibility. This VOI is analyzed using local phase analysis and the random sample consensus algorithm to refine the camera-estimated needle axis. The needle tip is determined by searching the localized needle axis using a probabilistic approach. Dynamic needle tracking in a sequence of 3D ultrasound volumes is enabled by iteratively applying a Kalman filter to estimate the VOI that includes the needle in the successive ultrasound volume and limiting the localization analysis to this VOI. A series of ex vivo animal experiments are conducted to evaluate the accuracy of needle localization and tracking. The results show that the proposed method can localize the needle in individual ultrasound volumes with maximum error rates of 0.7 mm for the needle axis, 1.7° for the needle angle, and 1.2 mm for the needle tip. Moreover, the proposed method can track the needle in a sequence of ultrasound volumes with maximum error rates of 1.0 mm for the needle axis, 2.0° for the needle angle, and 1.7 mm for the needle tip. These results suggest the feasibility of applying the proposed method to localize and track the needle using 3D motorized curvilinear ultrasound probes

An Adaptive Algorithm to Identify Ambiguous Prostate Capsule Boundary Lines for Three-Dimensional Reconstruction and Quantitation

Currently there are few parameters that are used to compare the efficiency of different methods of cancerous prostate surgical removal. An accurate assessment of the percentage and depth of extra-capsular soft tissue removed with the prostate by the various surgical techniques can help surgeons determine the appropriateness of surgical approaches. Additionally, an objective assessment can allow a particular surgeon to compare individual performance against a standard. In order to facilitate 3D reconstruction and objective analysis and thus provide more accurate quantitation results when analyzing specimens, it is essential to automatically identify the capsule line that separates the prostate gland tissue from its extra-capsular tissue. However the prostate capsule is sometimes unrecognizable due to the naturally occurring intrusion of muscle and connective tissue into the prostate gland. At these regions where the capsule disappears, its contour can be arbitrarily reconstructed by drawing a continuing contour line based on the natural shape of the prostate gland. Presented here is a mathematical model that can be used in deciding the missing part of the capsule. This model approximates the missing parts of the capsule where it disappears to a standard shape by using a Generalized Hough Transform (GHT) approach to detect the prostate capsule. We also present an algorithm based on a least squares curve fitting technique that uses a prostate shape equation to merge previously detected capsule parts with the curve equation to produce an approximated curve that represents the prostate capsule. We have tested our algorithms using three shapes on 13 prostate slices that are cut at different locations from the apex and the results are promisin