A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

Symmetric Biomechanically Guided Prone-to-Supine Breast Image Registration

Prone-to-supine breast image registration has potential application in the fields of surgical and radiotherapy planning, image guided interventions, and multi-modal cancer diagnosis, staging, and therapy response prediction. However, breast image registration of three dimensional images acquired in different patient positions is a challenging problem, due to large deformations induced to the soft breast tissue caused by the change in gravity loading. We present a symmetric, biomechanical simulation based registration framework which aligns the images in a central, virtually unloaded configuration. The breast tissue is modelled as a neo-Hookean material and gravity is considered as the main source of deformation in the original images. In addition to gravity, our framework successively applies image derived forces directly into the unloading simulation in place of a subsequent image registration step. This results in a biomechanically constrained deformation. Using a finite difference scheme avoids an explicit meshing step and enables simulations to be performed directly in the image space. The explicit time integration scheme allows the motion at the interface between chest and breast to be constrained along the chest wall. The feasibility and accuracy of the approach presented here was assessed by measuring the target registration error (TRE) using a numerical phantom with known ground truth deformations, nine clinical prone MRI and supine CT image pairs, one clinical prone-supine CT image pair and four prone-supine MRI image pairs. The registration reduced the mean TRE for the numerical phantom experiment from initially 19.3 to 0.9 mm and the combined mean TRE for all fourteen clinical data sets from 69.7 to 5.6 mm

Biomechanics-informed Neural Networks for Myocardial Motion Tracking in MRI

Image registration is an ill-posed inverse problem which often requires regularisation on the solution space. In contrast to most of the current approaches which impose explicit regularisation terms such as smoothness, in this paper we propose a novel method that can implicitly learn biomechanics-informed regularisation. Such an approach can incorporate application-specific prior knowledge into deep learning based registration. Particularly, the proposed biomechanics-informed regularisation leverages a variational autoencoder (VAE) to learn a manifold for biomechanically plausible deformations and to implicitly capture their underlying properties via reconstructing biomechanical simulations. The learnt VAE regulariser then can be coupled with any deep learning based registration network to regularise the solution space to be biomechanically plausible. The proposed method is validated in the context of myocardial motion tracking on 2D stacks of cardiac MRI data from two different datasets. The results show that it can achieve better performance against other competing methods in terms of motion tracking accuracy and has the ability to learn biomechanical properties such as incompressibility and strains. The method has also been shown to have better generalisability to unseen domains compared with commonly used L2 regularisation schemes.Comment: The paper is early accepted by MICCAI 202

Mechanical suppression of osteolytic bone metastases in advanced breast cancer patients: A randomised controlled study protocol evaluating safety, feasibility and preliminary efficacy of exercise as a targeted medicine

Background: Skeletal metastases present a major challenge for clinicians, representing an advanced and typically incurable stage of cancer. Bone is also the most common location for metastatic breast carcinoma, with skeletal lesions identified in over 80% of patients with advanced breast cancer. Preclinical models have demonstrated the ability of mechanical stimulation to suppress tumour formation and promote skeletal preservation at bone sites with osteolytic lesions, generating modulatory interference of tumour-driven bone remodelling. Preclinical studies have also demonstrated anti-cancer effects through exercise by minimising tumour hypoxia, normalising tumour vasculature and increasing tumoural blood perfusion. This study proposes to explore the promising role of targeted exercise to suppress tumour growth while concomitantly delivering broader health benefits in patients with advanced breast cancer with osteolytic bone metastases. Methods: This single-blinded, two-armed, randomised and controlled pilot study aims to establish the safety, feasibility and efficacy of an individually tailored, modular multi-modal exercise programme incorporating spinal isometric training (targeted muscle contraction) in 40 women with advanced breast cancer and stable osteolytic spinal metastases. Participants will be randomly assigned to exercise or usual medical care. The intervention arm will receive a 3-month clinically supervised exercise programme, which if proven to be safe and efficacious will be offered to the control-arm patients following study completion. Primary endpoints (programme feasibility, safety, tolerance and adherence) and secondary endpoints (tumour morphology, serum tumour biomarkers, bone metabolism, inflammation, anthropometry, body composition, bone pain, physical function and patient-reported outcomes) will be measured at baseline and following the intervention. Discussion: Exercise medicine may positively alter tumour biology through numerous mechanical and nonmechanical mechanisms. This randomised controlled pilot trial will explore the preliminary effects of targeted exercise on tumour morphology and circulating metastatic tumour biomarkers using an osteolytic skeletal metastases model in patients with breast cancer. The study is principally aimed at establishing feasibility and safety. If proven to be safe and feasible, results from this study could have important implications for the delivery of this exercise programme to patients with advanced cancer and sclerotic skeletal metastases or with skeletal lesions present in haematological cancers (such as osteolytic lesions in multiple myeloma), for which future research is recommended. Trial registration: anzctr.org.au, ACTRN-12616001368426. Registered on 4 October 2016

Visual perception driven registration of mammograms

International audienceThis paper aims to develop a methodology to register pairs of temporal mammograms. Control points based on anatomical features are detected in an automated way. Thereby, image semantic is used to extract landmarks based on these control points. A referential is generated from these control points based on this referential the studied images are realigned using different levels of observation leading to both rigid and non-rigid transforms according to expert mammogram reading

3D digital breast cancer models with multimodal fusion algorithms

Breast cancer image fusion consists of registering and visualizing different sets of a patient synchronized torso and radiological images into a 3D model. Breast spatial interpretation and visualization by the treating physician can be augmented with a patient-specific digital breast model that integrates radiological images. But the absence of a ground truth for a good correlation between surface and radiological information has impaired the development of potential clinical applications. A new image acquisition protocol was designed to acquire breast Magnetic Resonance Imaging (MRI) and 3D surface scan data with surface markers on the patient's breasts and torso. A patient-specific digital breast model integrating the real breast torso and the tumor location was created and validated with a MRI/3D surface scan fusion algorithm in 16 breast cancer patients. This protocol was used to quantify breast shape differences between different modalities, and to measure the target registration error of several variants of the MRI/3D scan fusion algorithm. The fusion of single breasts without the biomechanical model of pose transformation had acceptable registration errors and accurate tumor locations. The performance of the fusion algorithm was not affected by breast volume. Further research and virtual clinical interfaces could lead to fast integration of this fusion technology into clinical practice.publishersversionpublishe

Biomechanical modelling of probe to tissue interaction during ultrasound scanning

Purpose: Biomechanical simulation of anatomical deformations caused by ultrasound probe pressure is of outstanding importance for several applications, from the testing of robotic acquisition systems to multi-modal image fusion and development of ultrasound training platforms. Different approaches can be exploited for modelling the probe-tissue interaction, each achieving different trade-offs among accuracy, computation time and stability. Methods: We assess the performances of different strategies based on the finite element method for modelling the interaction between the rigid probe and soft tissues. Probe\u2013tissue contact is modelled using (i) penalty forces, (ii) constraint forces, and (iii) by prescribing the displacement of the mesh surface nodes. These methods are tested in the challenging context of ultrasound scanning of the breast, an organ undergoing large nonlinear deformations during the procedure. Results: The obtained results are evaluated against those of a non-physically based method. While all methods achieve similar accuracy, performance in terms of stability and speed shows high variability, especially for those methods modelling the contacts explicitly. Overall, prescribing surface displacements is the approach with best performances, but it requires prior knowledge of the contact area and probe trajectory. Conclusions: In this work, we present different strategies for modelling probe\u2013tissue interaction, each able to achieve different compromises among accuracy, speed and stability. The choice of the preferred approach highly depends on the requirements of the specific clinical application. Since the presented methodologies can be applied to describe general tool\u2013tissue interactions, this work can be seen as a reference for researchers seeking the most appropriate strategy to model anatomical deformation induced by the interaction with medical tools