Adversarial Deformation Regularization for Training Image Registration Neural Networks

We describe an adversarial learning approach to constrain convolutional neural network training for image registration, replacing heuristic smoothness measures of displacement fields often used in these tasks. Using minimally-invasive prostate cancer intervention as an example application, we demonstrate the feasibility of utilizing biomechanical simulations to regularize a weakly-supervised anatomical-label-driven registration network for aligning pre-procedural magnetic resonance (MR) and 3D intra-procedural transrectal ultrasound (TRUS) images. A discriminator network is optimized to distinguish the registration-predicted displacement fields from the motion data simulated by finite element analysis. During training, the registration network simultaneously aims to maximize similarity between anatomical labels that drives image alignment and to minimize an adversarial generator loss that measures divergence between the predicted- and simulated deformation. The end-to-end trained network enables efficient and fully-automated registration that only requires an MR and TRUS image pair as input, without anatomical labels or simulated data during inference. 108 pairs of labelled MR and TRUS images from 76 prostate cancer patients and 71,500 nonlinear finite-element simulations from 143 different patients were used for this study. We show that, with only gland segmentation as training labels, the proposed method can help predict physically plausible deformation without any other smoothness penalty. Based on cross-validation experiments using 834 pairs of independent validation landmarks, the proposed adversarial-regularized registration achieved a target registration error of 6.3 mm that is significantly lower than those from several other regularization methods.Comment: Accepted to MICCAI 201

Medical image computing and computer-aided medical interventions applied to soft tissues. Work in progress in urology

Until recently, Computer-Aided Medical Interventions (CAMI) and Medical Robotics have focused on rigid and non deformable anatomical structures. Nowadays, special attention is paid to soft tissues, raising complex issues due to their mobility and deformation. Mini-invasive digestive surgery was probably one of the first fields where soft tissues were handled through the development of simulators, tracking of anatomical structures and specific assistance robots. However, other clinical domains, for instance urology, are concerned. Indeed, laparoscopic surgery, new tumour destruction techniques (e.g. HIFU, radiofrequency, or cryoablation), increasingly early detection of cancer, and use of interventional and diagnostic imaging modalities, recently opened new challenges to the urologist and scientists involved in CAMI. This resulted in the last five years in a very significant increase of research and developments of computer-aided urology systems. In this paper, we propose a description of the main problems related to computer-aided diagnostic and therapy of soft tissues and give a survey of the different types of assistance offered to the urologist: robotization, image fusion, surgical navigation. Both research projects and operational industrial systems are discussed

Automatic Affine and Elastic Registration Strategies for Multi-dimensional Medical Images

Medical images have been used increasingly for diagnosis, treatment planning, monitoring disease processes, and other medical applications. A large variety of medical imaging modalities exists including CT, X-ray, MRI, Ultrasound, etc. Frequently a group of images need to be compared to one another and/or combined for research or cumulative purposes. In many medical studies, multiple images are acquired from subjects at different times or with different imaging modalities. Misalignment inevitably occurs, causing anatomical and/or functional feature shifts within the images. Computerized image registration (alignment) approaches can offer automatic and accurate image alignments without extensive user involvement and provide tools for visualizing combined images. This dissertation focuses on providing automatic image registration strategies. After a through review of existing image registration techniques, we identified two registration strategies that enhance the current field: (1) an automated rigid body and affine registration using voxel similarity measurements based on a sequential hybrid genetic algorithm, and (2) an automated deformable registration approach based upon a linear elastic finite element formulation. Both methods streamlined the registration process. They are completely automatic and require no user intervention. The proposed registration strategies were evaluated with numerous 2D and 3D MR images with a variety of tissue structures, orientations and dimensions. Multiple registration pathways were provided with guidelines for their applications. The sequential genetic algorithm mimics the pathway of an expert manually doing registration. Experiments demonstrated that the sequential genetic algorithm registration provides high alignment accuracy and is reliable for brain tissues. It avoids local minima/maxima traps of conventional optimization techniques, and does not require any preprocessing such as threshold, smoothing, segmentation, or definition of base points or edges. The elastic model was shown to be highly effective to accurately align areas of interest that are automatically extracted from the images, such as brains. Using a finite element method to get the displacement of each element node by applying a boundary mapping, this method provides an accurate image registration with excellent boundary alignment of each pair of slices and consequently align the entire volume automatically. This dissertation presented numerous volume alignments. Surface geometries were created directly from the aligned segmented images using the Multiple Material Marching Cubes algorithm. Using the proposed registration strategies, multiple subjects were aligned to a standard MRI reference, which is aligned to a segmented reference atlas. Consequently, multiple subjects are aligned to the segmented atlas and a full fMRI analysis is possible

Deformable mapping technique to correlate lesions in digital breast tomosynthesis and automated breast ultrasound images

Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/146324/1/mp13113.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/146324/2/mp13113_am.pd

A review of segmentation and deformable registration methods applied to adaptive cervical cancer radiation therapy treatment planning

Objective: Manual contouring and registration for radiotherapy treatment planning and online adaptation for cervical cancer radiation therapy in computed tomography (CT) and magnetic resonance images (MRI) are often necessary. However manual intervention is time consuming and may suffer from inter or intra-rater variability. In recent years a number of computer-guided automatic or semi-automatic segmentation and registration methods have been proposed. Segmentation and registration in CT and MRI for this purpose is a challenging task due to soft tissue deformation, inter-patient shape and appearance variation and anatomical changes over the course of treatment. The objective of this work is to provide a state-of-the-art review of computer-aided methods developed for adaptive treatment planning and radiation therapy planning for cervical cancer radiation therapy. Methods: Segmentation and registration methods published with the goal of cervical cancer treatment planning and adaptation have been identified from the literature (PubMed and Google Scholar). A comprehensive description of each method is provided. Similarities and differences of these methods are highlighted and the strengths and weaknesses of these methods are discussed. A discussion about choice of an appropriate method for a given modality is provided. Results: In the reviewed papers a Dice similarity coefficient of around 0.85 along with mean absolute surface distance of 2-4. mm for the clinically treated volume were reported for transfer of contours from planning day to the treatment day. Conclusions: Most segmentation and non-rigid registration methods have been primarily designed for adaptive re-planning for the transfer of contours from planning day to the treatment day. The use of shape priors significantly improved segmentation and registration accuracy compared to other models

Improving Dose-Response Correlations for Locally Advanced NSCLC Patients Treated with IMRT or PSPT

The standard of care for locally advanced non-small cell lung cancer (NSCLC) is concurrent chemo-radiotherapy. Despite recent advancements in radiation delivery methods, the median survival time of NSCLC patients remains below 28 months. Higher tumor dose has been found to increase survival but also a higher rate of radiation pneumonitis (RP) that affects breathing capability. In fear of such toxicity, less-aggressive treatment plans are often clinically preferred, leading to metastasis and recurrence. Therefore, accurate RP prediction is crucial to ensure tumor coverage to improve treatment outcome. Current models have associated RP with increased dose but with limited accuracy as they lack spatial correlation between accurate dose representation and quantitative RP representation. These models represent lung tissue damage with radiation dose distribution planned pre-treatment, which assumes a fixed patient geometry and inevitably renders imprecise dose delivery due to intra-fractional breathing motion and inter-fractional anatomy response. Additionally, current models employ whole-lung dose metrics as the contributing factor to RP as a qualitative, binary outcome but these global dose metrics discard microscopic, voxel-(3D pixel)-level information and prevent spatial correlations with quantitative RP representation. To tackle these limitations, we developed advanced deformable image registration (DIR) techniques that registered corresponding anatomical voxels between images for tracking and accumulating dose throughout treatment. DIR also enabled voxel-level dose-response correlation when CT image density change (IDC) was used to quantify RP. We hypothesized that more accurate estimates of biologically effective dose distributions actually delivered, achieved through (a) dose accumulation using deformable registration of weekly 4DCT images acquired over the course or radiotherapy and (b) the incorporation of variable relative biological effectiveness (RBE), would lead to statistically and clinically significant improvement in the correlation of RP with biologically effective dose distributions. Our work resulted in a robust intra-4DCT and inter-4DCT DIR workflow, with the accuracy meeting AAPM TG-132 recommendations for clinical implementation of DIR. The automated DIR workflow allowed us to develop a fully automated 4DCT-based dose accumulation pipeline in RayStation (RaySearch Laboratories, Stockholm, Sweden). With a sample of 67 IMRT patients, our results showed that the accumulated dose was statistically different than the planned dose across the entire cohort with an average MLD increase of ~1 Gy and clinically different for individual patients where 16% resulted in difference in the score of the normal tissue complication probability (NTCP) using an established, clinically used model, which could qualify the patients for treatment planning re-evaluation. Lastly, we associated dose difference with accuracy difference by establishing and comparing voxel-level dose-IDC correlations and concluded that the accumulated dose better described the localized damage, thereby a closer representation of the delivered dose. Using the same dose-response correlation strategy, we plotted the dose-IDC relationships for both photon patients (N = 51) and proton patients (N = 67), we measured the variable proton RBE values to be 3.07–1.27 from 9–52 Gy proton voxels. With the measured RBE values, we fitted an established variable proton RBE model with pseudo-R2 of 0.98. Therefore, our results led to statistically and clinically significant improvement in the correlation of RP with accumulated and biologically effective dose distributions and demonstrated the potential of incorporating the effect of anatomical change and biological damage in RP prediction models

Computational Anatomy for Multi-Organ Analysis in Medical Imaging: A Review

The medical image analysis field has traditionally been focused on the development of organ-, and disease-specific methods. Recently, the interest in the development of more 20 comprehensive computational anatomical models has grown, leading to the creation of multi-organ models. Multi-organ approaches, unlike traditional organ-specific strategies, incorporate inter-organ relations into the model, thus leading to a more accurate representation of the complex human anatomy. Inter-organ relations are not only spatial, but also functional and physiological. Over the years, the strategies 25 proposed to efficiently model multi-organ structures have evolved from the simple global modeling, to more sophisticated approaches such as sequential, hierarchical, or machine learning-based models. In this paper, we present a review of the state of the art on multi-organ analysis and associated computation anatomy methodology. The manuscript follows a methodology-based classification of the different techniques 30 available for the analysis of multi-organs and multi-anatomical structures, from techniques using point distribution models to the most recent deep learning-based approaches. With more than 300 papers included in this review, we reflect on the trends and challenges of the field of computational anatomy, the particularities of each anatomical region, and the potential of multi-organ analysis to increase the impact of 35 medical imaging applications on the future of healthcare.Comment: Paper under revie

Computer- and robot-assisted Medical Intervention

Medical robotics includes assistive devices used by the physician in order to make his/her diagnostic or therapeutic practices easier and more efficient. This chapter focuses on such systems. It introduces the general field of Computer-Assisted Medical Interventions, its aims, its different components and describes the place of robots in that context. The evolutions in terms of general design and control paradigms in the development of medical robots are presented and issues specific to that application domain are discussed. A view of existing systems, on-going developments and future trends is given. A case-study is detailed. Other types of robotic help in the medical environment (such as for assisting a handicapped person, for rehabilitation of a patient or for replacement of some damaged/suppressed limbs or organs) are out of the scope of this chapter.Comment: Handbook of Automation, Shimon Nof (Ed.) (2009) 000-00