363 research outputs found
Learning Deep Similarity Metric for 3D MR-TRUS Registration
Purpose: The fusion of transrectal ultrasound (TRUS) and magnetic resonance
(MR) images for guiding targeted prostate biopsy has significantly improved the
biopsy yield of aggressive cancers. A key component of MR-TRUS fusion is image
registration. However, it is very challenging to obtain a robust automatic
MR-TRUS registration due to the large appearance difference between the two
imaging modalities. The work presented in this paper aims to tackle this
problem by addressing two challenges: (i) the definition of a suitable
similarity metric and (ii) the determination of a suitable optimization
strategy.
Methods: This work proposes the use of a deep convolutional neural network to
learn a similarity metric for MR-TRUS registration. We also use a composite
optimization strategy that explores the solution space in order to search for a
suitable initialization for the second-order optimization of the learned
metric. Further, a multi-pass approach is used in order to smooth the metric
for optimization.
Results: The learned similarity metric outperforms the classical mutual
information and also the state-of-the-art MIND feature based methods. The
results indicate that the overall registration framework has a large capture
range. The proposed deep similarity metric based approach obtained a mean TRE
of 3.86mm (with an initial TRE of 16mm) for this challenging problem.
Conclusion: A similarity metric that is learned using a deep neural network
can be used to assess the quality of any given image registration and can be
used in conjunction with the aforementioned optimization framework to perform
automatic registration that is robust to poor initialization.Comment: To appear on IJCAR
Medical image computing and computer-aided medical interventions applied to soft tissues. Work in progress in urology
Until recently, Computer-Aided Medical Interventions (CAMI) and Medical
Robotics have focused on rigid and non deformable anatomical structures.
Nowadays, special attention is paid to soft tissues, raising complex issues due
to their mobility and deformation. Mini-invasive digestive surgery was probably
one of the first fields where soft tissues were handled through the development
of simulators, tracking of anatomical structures and specific assistance
robots. However, other clinical domains, for instance urology, are concerned.
Indeed, laparoscopic surgery, new tumour destruction techniques (e.g. HIFU,
radiofrequency, or cryoablation), increasingly early detection of cancer, and
use of interventional and diagnostic imaging modalities, recently opened new
challenges to the urologist and scientists involved in CAMI. This resulted in
the last five years in a very significant increase of research and developments
of computer-aided urology systems. In this paper, we propose a description of
the main problems related to computer-aided diagnostic and therapy of soft
tissues and give a survey of the different types of assistance offered to the
urologist: robotization, image fusion, surgical navigation. Both research
projects and operational industrial systems are discussed
Label-driven weakly-supervised learning for multimodal deformable image registration
Spatially aligning medical images from different modalities remains a
challenging task, especially for intraoperative applications that require fast
and robust algorithms. We propose a weakly-supervised, label-driven formulation
for learning 3D voxel correspondence from higher-level label correspondence,
thereby bypassing classical intensity-based image similarity measures. During
training, a convolutional neural network is optimised by outputting a dense
displacement field (DDF) that warps a set of available anatomical labels from
the moving image to match their corresponding counterparts in the fixed image.
These label pairs, including solid organs, ducts, vessels, point landmarks and
other ad hoc structures, are only required at training time and can be
spatially aligned by minimising a cross-entropy function of the warped moving
label and the fixed label. During inference, the trained network takes a new
image pair to predict an optimal DDF, resulting in a fully-automatic,
label-free, real-time and deformable registration. For interventional
applications where large global transformation prevails, we also propose a
neural network architecture to jointly optimise the global- and local
displacements. Experiment results are presented based on cross-validating
registrations of 111 pairs of T2-weighted magnetic resonance images and 3D
transrectal ultrasound images from prostate cancer patients with a total of
over 4000 anatomical labels, yielding a median target registration error of 4.2
mm on landmark centroids and a median Dice of 0.88 on prostate glands.Comment: Accepted to ISBI 201
Registration of magnetic resonance and ultrasound images for guiding prostate cancer interventions
Prostate cancer is a major international health problem with a large and rising incidence in many parts of the world. Transrectal ultrasound (TRUS) imaging is used routinely to guide surgical procedures, such as needle biopsy and a number of minimally-invasive therapies, but its limited ability to visualise prostate cancer is widely recognised. Magnetic resonance (MR) imaging techniques, on the other hand, have recently been developed that can provide clinically useful diagnostic information. Registration (or alignment) of MR and TRUS images during TRUS-guided surgical interventions potentially provides a cost-effective approach to augment TRUS images with clinically useful, MR-derived information (for example, tumour location, shape and size). This thesis describes a deformable image registration framework that enables automatic and/or semi-automatic alignment of MR and 3D TRUS images of the prostate gland. The method combines two technical developments in the field: First, a method for constructing patient-specific statistical shape models of prostate motion/deformation, based on learning from finite element simulations of gland motion using geometric data from a preoperative MR image, is proposed. Second, a novel “model-to-image” registration framework is developed to register this statistical shape model automatically to an intraoperative TRUS image. This registration approach is implemented using a novel model-to-image vector alignment (MIVA) algorithm, which maximises the likelihood of a particular instance of a statistical shape model given a voxel-intensity-based feature vector that represents an estimate of the surface normal vectors at the boundary of the organ in question. Using real patient data, the MR-TRUS registration accuracy of the new algorithm is validated using intra-prostatic anatomical landmarks. A rigorous and extensive validation analysis is also provided for assessing the image registration experiments. The final target registration error after performing 100 MR–TRUS registrations for each patient have a median of 2.40 mm, meaning that over 93% registrations may successfully hit the target representing a clinically significant lesion. The implemented registration algorithms took less than 30 seconds and 2 minutes for manually defined point- and normal vector features, respectively. The thesis concludes with a summary of potential applications and future research directions
Adversarial Deformation Regularization for Training Image Registration Neural Networks
We describe an adversarial learning approach to constrain convolutional
neural network training for image registration, replacing heuristic smoothness
measures of displacement fields often used in these tasks. Using
minimally-invasive prostate cancer intervention as an example application, we
demonstrate the feasibility of utilizing biomechanical simulations to
regularize a weakly-supervised anatomical-label-driven registration network for
aligning pre-procedural magnetic resonance (MR) and 3D intra-procedural
transrectal ultrasound (TRUS) images. A discriminator network is optimized to
distinguish the registration-predicted displacement fields from the motion data
simulated by finite element analysis. During training, the registration network
simultaneously aims to maximize similarity between anatomical labels that
drives image alignment and to minimize an adversarial generator loss that
measures divergence between the predicted- and simulated deformation. The
end-to-end trained network enables efficient and fully-automated registration
that only requires an MR and TRUS image pair as input, without anatomical
labels or simulated data during inference. 108 pairs of labelled MR and TRUS
images from 76 prostate cancer patients and 71,500 nonlinear finite-element
simulations from 143 different patients were used for this study. We show that,
with only gland segmentation as training labels, the proposed method can help
predict physically plausible deformation without any other smoothness penalty.
Based on cross-validation experiments using 834 pairs of independent validation
landmarks, the proposed adversarial-regularized registration achieved a target
registration error of 6.3 mm that is significantly lower than those from
several other regularization methods.Comment: Accepted to MICCAI 201
Image-based registration methods for quantification and compensation of prostate motion during trans-rectal ultrasound (TRUS)-guided biopsy
Prostate biopsy is the clinical standard for cancer diagnosis and is typically performed under two-dimensional (2D) transrectal ultrasound (TRUS) for needle guidance. Unfortunately, most early stage prostate cancers are not visible on ultrasound and the procedure suffers from high false negative rates due to the lack of visible targets. Fusion of pre-biopsy MRI to 3D TRUS for targeted biopsy could improve cancer detection rates and volume of tumor sampled. In MRI-TRUS fusion biopsy systems, patient or prostate motion during the procedure causes misalignments in the MR targets mapped to the live 2D TRUS images, limiting the targeting accuracy of the biopsy system.
In order to sample smallest clinically significant tumours of 0.5 cm3with 95% confidence, the root mean square (RMS) error of the biopsy system needs to be
The target misalignments due to intermittent prostate motion during the procedure can be compensated by registering the live 2D TRUS images acquired during the biopsy procedure to the pre-acquired baseline 3D TRUS image. The registration must be performed both accurately and quickly in order to be useful during the clinical procedure. We developed an intensity-based 2D-3D rigid registration algorithm and validated it by calculating the target registration error (TRE) using manually identified fiducials within the prostate. We discuss two different approaches that can be used to improve the robustness of this registration to meet the clinical requirements. Firstly, we evaluated the impact of intra-procedural 3D TRUS imaging on motion compensation accuracy since the limited anatomical context available in live 2D TRUS images could limit the robustness of the 2D-3D registration. The results indicated that TRE improved when intra-procedural 3D TRUS images were used in registration, with larger improvements in the base and apex regions as compared with the mid-gland region. Secondly, we developed and evaluated a registration algorithm whose optimization is based on learned prostate motion characteristics. Compared to our initial approach, the updated optimization improved the robustness during 2D-3D registration by reducing the number of registrations with a TRE \u3e 5 mm from 9.2% to 1.2% with an overall RMS TRE of 2.3 mm.
The methods developed in this work were intended to improve the needle targeting accuracy of 3D TRUS-guided biopsy systems. The successful integration of the techniques into current 3D TRUS-guided systems could improve the overall cancer detection rate during the biopsy and help to achieve earlier diagnosis and fewer repeat biopsy procedures in prostate cancer diagnosis
Meta-Learning Initializations for Interactive Medical Image Registration
We present a meta-learning framework for interactive medical image
registration. Our proposed framework comprises three components: a
learning-based medical image registration algorithm, a form of user interaction
that refines registration at inference, and a meta-learning protocol that
learns a rapidly adaptable network initialization. This paper describes a
specific algorithm that implements the registration, interaction and
meta-learning protocol for our exemplar clinical application: registration of
magnetic resonance (MR) imaging to interactively acquired, sparsely-sampled
transrectal ultrasound (TRUS) images. Our approach obtains comparable
registration error (4.26 mm) to the best-performing non-interactive
learning-based 3D-to-3D method (3.97 mm) while requiring only a fraction of the
data, and occurring in real-time during acquisition. Applying sparsely sampled
data to non-interactive methods yields higher registration errors (6.26 mm),
demonstrating the effectiveness of interactive MR-TRUS registration, which may
be applied intraoperatively given the real-time nature of the adaptation
process.Comment: 11 pages, 10 figures. Paper accepted to IEEE Transactions on Medical
Imaging (October 26 2022
Real-time multimodal image registration with partial intraoperative point-set data
We present Free Point Transformer (FPT) - a deep neural network architecture for non-rigid point-set registration. Consisting of two modules, a global feature extraction module and a point transformation module, FPT does not assume explicit constraints based on point vicinity, thereby overcoming a common requirement of previous learning-based point-set registration methods. FPT is designed to accept unordered and unstructured point-sets with a variable number of points and uses a "model-free" approach without heuristic constraints. Training FPT is flexible and involves minimizing an intuitive unsupervised loss function, but supervised, semi-supervised, and partially- or weakly-supervised training are also supported. This flexibility makes FPT amenable to multimodal image registration problems where the ground-truth deformations are difficult or impossible to measure. In this paper, we demonstrate the application of FPT to non-rigid registration of prostate magnetic resonance (MR) imaging and sparsely-sampled transrectal ultrasound (TRUS) images. The registration errors were 4.71 mm and 4.81 mm for complete TRUS imaging and sparsely-sampled TRUS imaging, respectively. The results indicate superior accuracy to the alternative rigid and non-rigid registration algorithms tested and substantially lower computation time. The rapid inference possible with FPT makes it particularly suitable for applications where real-time registration is beneficial
New Mechatronic Systems for the Diagnosis and Treatment of Cancer
Both two dimensional (2D) and three dimensional (3D) imaging modalities are useful tools for viewing the internal anatomy. Three dimensional imaging techniques are required for accurate targeting of needles. This improves the efficiency and control over the intervention as the high temporal resolution of medical images can be used to validate the location of needle and target in real time. Relying on imaging alone, however, means the intervention is still operator dependent because of the difficulty of controlling the location of the needle within the image. The objective of this thesis is to improve the accuracy and repeatability of needle-based interventions over conventional techniques: both manual and automated techniques. This includes increasing the accuracy and repeatability of these procedures in order to minimize the invasiveness of the procedure.
In this thesis, I propose that by combining the remote center of motion concept using spherical linkage components into a passive or semi-automated device, the physician will have a useful tracking and guidance system at their disposal in a package, which is less threatening than a robot to both the patient and physician. This design concept offers both the manipulative transparency of a freehand system, and tremor reduction through scaling currently offered in automated systems. In addressing each objective of this thesis, a number of novel mechanical designs incorporating an remote center of motion architecture with varying degrees of freedom have been presented. Each of these designs can be deployed in a variety of imaging modalities and clinical applications, ranging from preclinical to human interventions, with an accuracy of control in the millimeter to sub-millimeter range
Multimodality Biomedical Image Registration using Free Point Transformer Networks
We describe a point-set registration algorithm based on a novel free point
transformer (FPT) network, designed for points extracted from multimodal
biomedical images for registration tasks, such as those frequently encountered
in ultrasound-guided interventional procedures. FPT is constructed with a
global feature extractor which accepts unordered source and target point-sets
of variable size. The extracted features are conditioned by a shared multilayer
perceptron point transformer module to predict a displacement vector for each
source point, transforming it into the target space. The point transformer
module assumes no vicinity or smoothness in predicting spatial transformation
and, together with the global feature extractor, is trained in a data-driven
fashion with an unsupervised loss function. In a multimodal registration task
using prostate MR and sparsely acquired ultrasound images, FPT yields
comparable or improved results over other rigid and non-rigid registration
methods. This demonstrates the versatility of FPT to learn registration
directly from real, clinical training data and to generalize to a challenging
task, such as the interventional application presented.Comment: 10 pages, 4 figures. Accepted for publication at International
Conference on Medical Image Computing and Computer Assisted Intervention
(MICCAI) workshop on Advances in Simplifying Medical UltraSound (ASMUS) 202
- …