Predicting Slice-to-Volume Transformation in Presence of Arbitrary Subject Motion

This paper aims to solve a fundamental problem in intensity-based 2D/3D registration, which concerns the limited capture range and need for very good initialization of state-of-the-art image registration methods. We propose a regression approach that learns to predict rotation and translations of arbitrary 2D image slices from 3D volumes, with respect to a learned canonical atlas co-ordinate system. To this end, we utilize Convolutional Neural Networks (CNNs) to learn the highly complex regression function that maps 2D image slices into their correct position and orientation in 3D space. Our approach is attractive in challenging imaging scenarios, where significant subject motion complicates reconstruction performance of 3D volumes from 2D slice data. We extensively evaluate the effectiveness of our approach quantitatively on simulated MRI brain data with extreme random motion. We further demonstrate qualitative results on fetal MRI where our method is integrated into a full reconstruction and motion compensation pipeline. With our CNN regression approach we obtain an average prediction error of 7mm on simulated data, and convincing reconstruction quality of images of very young fetuses where previous methods fail. We further discuss applications to Computed Tomography and X-ray projections. Our approach is a general solution to the 2D/3D initialization problem. It is computationally efficient, with prediction times per slice of a few milliseconds, making it suitable for real-time scenarios.Comment: 8 pages, 4 figures, 6 pages supplemental material, currently under review for MICCAI 201

Automatic 3D bi-ventricular segmentation of cardiac images by a shape-refined multi-task deep learning approach

Deep learning approaches have achieved state-of-the-art performance in cardiac magnetic resonance (CMR) image segmentation. However, most approaches have focused on learning image intensity features for segmentation, whereas the incorporation of anatomical shape priors has received less attention. In this paper, we combine a multi-task deep learning approach with atlas propagation to develop a shape-constrained bi-ventricular segmentation pipeline for short-axis CMR volumetric images. The pipeline first employs a fully convolutional network (FCN) that learns segmentation and landmark localisation tasks simultaneously. The architecture of the proposed FCN uses a 2.5D representation, thus combining the computational advantage of 2D FCNs networks and the capability of addressing 3D spatial consistency without compromising segmentation accuracy. Moreover, the refinement step is designed to explicitly enforce a shape constraint and improve segmentation quality. This step is effective for overcoming image artefacts (e.g. due to different breath-hold positions and large slice thickness), which preclude the creation of anatomically meaningful 3D cardiac shapes. The proposed pipeline is fully automated, due to network's ability to infer landmarks, which are then used downstream in the pipeline to initialise atlas propagation. We validate the pipeline on 1831 healthy subjects and 649 subjects with pulmonary hypertension. Extensive numerical experiments on the two datasets demonstrate that our proposed method is robust and capable of producing accurate, high-resolution and anatomically smooth bi-ventricular 3D models, despite the artefacts in input CMR volumes

Label-driven weakly-supervised learning for multimodal deformable image registration

Spatially aligning medical images from different modalities remains a challenging task, especially for intraoperative applications that require fast and robust algorithms. We propose a weakly-supervised, label-driven formulation for learning 3D voxel correspondence from higher-level label correspondence, thereby bypassing classical intensity-based image similarity measures. During training, a convolutional neural network is optimised by outputting a dense displacement field (DDF) that warps a set of available anatomical labels from the moving image to match their corresponding counterparts in the fixed image. These label pairs, including solid organs, ducts, vessels, point landmarks and other ad hoc structures, are only required at training time and can be spatially aligned by minimising a cross-entropy function of the warped moving label and the fixed label. During inference, the trained network takes a new image pair to predict an optimal DDF, resulting in a fully-automatic, label-free, real-time and deformable registration. For interventional applications where large global transformation prevails, we also propose a neural network architecture to jointly optimise the global- and local displacements. Experiment results are presented based on cross-validating registrations of 111 pairs of T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients with a total of over 4000 anatomical labels, yielding a median target registration error of 4.2 mm on landmark centroids and a median Dice of 0.88 on prostate glands.Comment: Accepted to ISBI 201

Endoluminal surface registration for CT colonography using Haustral Fold Matching

Computed Tomographic (CT) colonography is a technique used for the detection of bowel cancer or potentially precancerous polyps. The procedure is performed routinely with the patient both prone and supine to differentiate fixed colonic pathology from mobile faecal residue. Matching corresponding locations is difficult and time consuming for radiologists due to colonic deformations that occur during patient repositioning. We propose a novel method to establish correspondence between the two acquisitions automatically. The problem is first simplified by detecting haustral folds using a graph cut method applied to a curvature-based metric applied to a surface mesh generated from segmentation of the colonic lumen. A virtual camera is used to create a set of images that provide a metric for matching pairs of folds between the prone and supine acquisitions. Image patches are generated at the fold positions using depth map renderings of the endoluminal surface and optimised by performing a virtual camera registration over a restricted set of degrees of freedom. The intensity difference between image pairs, along with additional neighbourhood information to enforce geometric constraints over a 2D parameterisation of the 3D space, are used as unary and pair-wise costs respectively, and included in a Markov Random Field (MRF) model to estimate the maximum a-posteriori fold labelling assignment. The method achieved fold matching accuracy of 96.0% and 96.1% in patient cases with and without local colonic collapse. Moreover, it improved upon an existing surface-based registration algorithm by providing an initialisation. The set of landmark correspondences is used to non-rigidly transform a 2D source image derived from a conformal mapping process on the 3D endoluminal surface mesh. This achieves full surface correspondence between prone and supine views and can be further refined with an intensity based registration showing a statistically significant improvement (p<0.001p<0.001), and decreasing mean error from 11.9mm11.9mm to 6.0mm6.0mm measured at 1743 reference points from 17 CTC datasets

Prostate MR image segmentation using 3D active appearance models

This paper presents a method for automatic segmentation of the prostate from transversal T2-weighted images based on 3D Active Appearance Models (AAM). The algorithm consist of two stages. Firstly, Shape Context based non-rigid surface registration of the manual segmented images is used to obtain the point correspondence between the given training cases. Subsequently, an AAM is used to segment the prostate on 50 training cases. The method is evaluated using a 5-fold cross validation over 5 repetitions. The mean Dice similarity coefficient and 95% Hausdorff distance are 0.78 and 7.32 mm respectively

Blood Vessel Segmentation and shape analysis for quantification of Coronary Artery Stenosis in CT Angiography

This thesis presents an automated framework for quantitative vascular shape analysis of the coronary arteries, which constitutes an important and fundamental component of an automated image-based diagnostic system. Firstly, an automated vessel segmentation algorithm is developed to extract the coronary arteries based on the framework of active contours. Both global and local intensity statistics are utilised in the energy functional calculation, which allows for dealing with non-uniform brightness conditions, while evolving the contour towards to the desired boundaries without being trapped in local minima. To suppress kissing vessel artifacts, a slice-by-slice correction scheme, based on multiple regions competition, is proposed to identify and track the kissing vessels throughout the transaxial images of the CTA data. Based on the resulting segmentation, we then present a dedicated algorithm to estimate the geometric parameters of the extracted arteries, with focus on vessel bifurcations. In particular, the centreline and associated reference surface of the coronary arteries, in the vicinity of arterial bifurcations, are determined by registering an elliptical cross sectional tube to the desired constituent branch. The registration problem is solved by a hybrid optimisation method, combining local greedy search and dynamic programming, which ensures the global optimality of the solution and permits the incorporation of any hard constraints posed to the tube model within a natural and direct framework. In contrast with conventional volume domain methods, this technique works directly on the mesh domain, thus alleviating the need for image upsampling. The performance of the proposed framework, in terms of efficiency and accuracy, is demonstrated on both synthetic and clinical image data. Experimental results have shown that our techniques are capable of extracting the major branches of the coronary arteries and estimating the related geometric parameters (i.e., the centreline and the reference surface) with a high degree of agreement to those obtained through manual delineation. Particularly, all of the major branches of coronary arteries are successfully detected by the proposed technique, with a voxel-wise error at 0.73 voxels to the manually delineated ground truth data. Through the application of the slice-by-slice correction scheme, the false positive metric, for those coronary segments affected by kissing vessel artifacts, reduces from 294% to 22.5%. In terms of the capability of the presented framework in defining the location of centrelines across vessel bifurcations, the mean square errors (MSE) of the resulting centreline, with respect to the ground truth data, is reduced by an average of 62.3%, when compared with initial estimation obtained using a topological thinning based algorithm

Automated segmentation and analysis of normal and osteoarthritic knee menisci from magnetic resonance images: data from the Osteoarthritis Initiative

OBJECTIVE: To validate an automatic scheme for the segmentation and quantitative analysis of the medial meniscus (MM) and lateral meniscus (LM) in magnetic resonance (MR) images of the knee

Deep learning-based plane pose regression in obstetric ultrasound

PURPOSE: In obstetric ultrasound (US) scanning, the learner's ability to mentally build a three-dimensional (3D) map of the fetus from a two-dimensional (2D) US image represents a major challenge in skill acquisition. We aim to build a US plane localisation system for 3D visualisation, training, and guidance without integrating additional sensors. METHODS: We propose a regression convolutional neural network (CNN) using image features to estimate the six-dimensional pose of arbitrarily oriented US planes relative to the fetal brain centre. The network was trained on synthetic images acquired from phantom 3D US volumes and fine-tuned on real scans. Training data was generated by slicing US volumes into imaging planes in Unity at random coordinates and more densely around the standard transventricular (TV) plane. RESULTS: With phantom data, the median errors are 0.90 mm/1.17[Formula: see text] and 0.44 mm/1.21[Formula: see text] for random planes and planes close to the TV one, respectively. With real data, using a different fetus with the same gestational age (GA), these errors are 11.84 mm/25.17[Formula: see text]. The average inference time is 2.97 ms per plane. CONCLUSION: The proposed network reliably localises US planes within the fetal brain in phantom data and successfully generalises pose regression for an unseen fetal brain from a similar GA as in training. Future development will expand the prediction to volumes of the whole fetus and assess its potential for vision-based, freehand US-assisted navigation when acquiring standard fetal planes

Automated pick-up of suturing needles for robotic surgical assistance

Robot-assisted laparoscopic prostatectomy (RALP) is a treatment for prostate cancer that involves complete or nerve sparing removal prostate tissue that contains cancer. After removal the bladder neck is successively sutured directly with the urethra. The procedure is called urethrovesical anastomosis and is one of the most dexterity demanding tasks during RALP. Two suturing instruments and a pair of needles are used in combination to perform a running stitch during urethrovesical anastomosis. While robotic instruments provide enhanced dexterity to perform the anastomosis, it is still highly challenging and difficult to learn. In this paper, we presents a vision-guided needle grasping method for automatically grasping the needle that has been inserted into the patient prior to anastomosis. We aim to automatically grasp the suturing needle in a position that avoids hand-offs and immediately enables the start of suturing. The full grasping process can be broken down into: a needle detection algorithm; an approach phase where the surgical tool moves closer to the needle based on visual feedback; and a grasping phase through path planning based on observed surgical practice. Our experimental results show examples of successful autonomous grasping that has the potential to simplify and decrease the operational time in RALP by assisting a small component of urethrovesical anastomosis