896 research outputs found
Simulating Patho-realistic Ultrasound Images using Deep Generative Networks with Adversarial Learning
Ultrasound imaging makes use of backscattering of waves during their
interaction with scatterers present in biological tissues. Simulation of
synthetic ultrasound images is a challenging problem on account of inability to
accurately model various factors of which some include intra-/inter scanline
interference, transducer to surface coupling, artifacts on transducer elements,
inhomogeneous shadowing and nonlinear attenuation. Current approaches typically
solve wave space equations making them computationally expensive and slow to
operate. We propose a generative adversarial network (GAN) inspired approach
for fast simulation of patho-realistic ultrasound images. We apply the
framework to intravascular ultrasound (IVUS) simulation. A stage 0 simulation
performed using pseudo B-mode ultrasound image simulator yields speckle mapping
of a digitally defined phantom. The stage I GAN subsequently refines them to
preserve tissue specific speckle intensities. The stage II GAN further refines
them to generate high resolution images with patho-realistic speckle profiles.
We evaluate patho-realism of simulated images with a visual Turing test
indicating an equivocal confusion in discriminating simulated from real. We
also quantify the shift in tissue specific intensity distributions of the real
and simulated images to prove their similarity.Comment: To appear in the Proceedings of the 2018 IEEE International Symposium
on Biomedical Imaging (ISBI 2018
Adversarial Deformation Regularization for Training Image Registration Neural Networks
We describe an adversarial learning approach to constrain convolutional
neural network training for image registration, replacing heuristic smoothness
measures of displacement fields often used in these tasks. Using
minimally-invasive prostate cancer intervention as an example application, we
demonstrate the feasibility of utilizing biomechanical simulations to
regularize a weakly-supervised anatomical-label-driven registration network for
aligning pre-procedural magnetic resonance (MR) and 3D intra-procedural
transrectal ultrasound (TRUS) images. A discriminator network is optimized to
distinguish the registration-predicted displacement fields from the motion data
simulated by finite element analysis. During training, the registration network
simultaneously aims to maximize similarity between anatomical labels that
drives image alignment and to minimize an adversarial generator loss that
measures divergence between the predicted- and simulated deformation. The
end-to-end trained network enables efficient and fully-automated registration
that only requires an MR and TRUS image pair as input, without anatomical
labels or simulated data during inference. 108 pairs of labelled MR and TRUS
images from 76 prostate cancer patients and 71,500 nonlinear finite-element
simulations from 143 different patients were used for this study. We show that,
with only gland segmentation as training labels, the proposed method can help
predict physically plausible deformation without any other smoothness penalty.
Based on cross-validation experiments using 834 pairs of independent validation
landmarks, the proposed adversarial-regularized registration achieved a target
registration error of 6.3 mm that is significantly lower than those from
several other regularization methods.Comment: Accepted to MICCAI 201
Intraoperative Organ Motion Models with an Ensemble of Conditional Generative Adversarial Networks
In this paper, we describe how a patient-specific, ultrasound-probe-induced
prostate motion model can be directly generated from a single preoperative MR
image. Our motion model allows for sampling from the conditional distribution
of dense displacement fields, is encoded by a generative neural network
conditioned on a medical image, and accepts random noise as additional input.
The generative network is trained by a minimax optimisation with a second
discriminative neural network, tasked to distinguish generated samples from
training motion data. In this work, we propose that 1) jointly optimising a
third conditioning neural network that pre-processes the input image, can
effectively extract patient-specific features for conditioning; and 2)
combining multiple generative models trained separately with heuristically
pre-disjointed training data sets can adequately mitigate the problem of mode
collapse. Trained with diagnostic T2-weighted MR images from 143 real patients
and 73,216 3D dense displacement fields from finite element simulations of
intraoperative prostate motion due to transrectal ultrasound probe pressure,
the proposed models produced physically-plausible patient-specific motion of
prostate glands. The ability to capture biomechanically simulated motion was
evaluated using two errors representing generalisability and specificity of the
model. The median values, calculated from a 10-fold cross-validation, were
2.8+/-0.3 mm and 1.7+/-0.1 mm, respectively. We conclude that the introduced
approach demonstrates the feasibility of applying state-of-the-art machine
learning algorithms to generate organ motion models from patient images, and
shows significant promise for future research.Comment: Accepted to MICCAI 201
NiftyNet: a deep-learning platform for medical imaging
Medical image analysis and computer-assisted intervention problems are
increasingly being addressed with deep-learning-based solutions. Established
deep-learning platforms are flexible but do not provide specific functionality
for medical image analysis and adapting them for this application requires
substantial implementation effort. Thus, there has been substantial duplication
of effort and incompatible infrastructure developed across many research
groups. This work presents the open-source NiftyNet platform for deep learning
in medical imaging. The ambition of NiftyNet is to accelerate and simplify the
development of these solutions, and to provide a common mechanism for
disseminating research outputs for the community to use, adapt and build upon.
NiftyNet provides a modular deep-learning pipeline for a range of medical
imaging applications including segmentation, regression, image generation and
representation learning applications. Components of the NiftyNet pipeline
including data loading, data augmentation, network architectures, loss
functions and evaluation metrics are tailored to, and take advantage of, the
idiosyncracies of medical image analysis and computer-assisted intervention.
NiftyNet is built on TensorFlow and supports TensorBoard visualization of 2D
and 3D images and computational graphs by default.
We present 3 illustrative medical image analysis applications built using
NiftyNet: (1) segmentation of multiple abdominal organs from computed
tomography; (2) image regression to predict computed tomography attenuation
maps from brain magnetic resonance images; and (3) generation of simulated
ultrasound images for specified anatomical poses.
NiftyNet enables researchers to rapidly develop and distribute deep learning
solutions for segmentation, regression, image generation and representation
learning applications, or extend the platform to new applications.Comment: Wenqi Li and Eli Gibson contributed equally to this work. M. Jorge
Cardoso and Tom Vercauteren contributed equally to this work. 26 pages, 6
figures; Update includes additional applications, updated author list and
formatting for journal submissio
Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks
Sonography synthesis has a wide range of applications, including medical
procedure simulation, clinical training and multimodality image registration.
In this paper, we propose a machine learning approach to simulate ultrasound
images at given 3D spatial locations (relative to the patient anatomy), based
on conditional generative adversarial networks (GANs). In particular, we
introduce a novel neural network architecture that can sample anatomically
accurate images conditionally on spatial position of the (real or mock)
freehand ultrasound probe. To ensure an effective and efficient spatial
information assimilation, the proposed spatially-conditioned GANs take
calibrated pixel coordinates in global physical space as conditioning input,
and utilise residual network units and shortcuts of conditioning data in the
GANs' discriminator and generator, respectively. Using optically tracked B-mode
ultrasound images, acquired by an experienced sonographer on a fetus phantom,
we demonstrate the feasibility of the proposed method by two sets of
quantitative results: distances were calculated between corresponding
anatomical landmarks identified in the held-out ultrasound images and the
simulated data at the same locations unseen to the networks; a usability study
was carried out to distinguish the simulated data from the real images. In
summary, we present what we believe are state-of-the-art visually realistic
ultrasound images, simulated by the proposed GAN architecture that is stable to
train and capable of generating plausibly diverse image samples.Comment: Accepted to MICCAI RAMBO 201
Constrained CycleGAN for Effective Generation of Ultrasound Sector Images of Improved Spatial Resolution
Objective. A phased or a curvilinear array produces ultrasound (US) images
with a sector field of view (FOV), which inherently exhibits spatially-varying
image resolution with inferior quality in the far zone and towards the two
sides azimuthally. Sector US images with improved spatial resolutions are
favorable for accurate quantitative analysis of large and dynamic organs, such
as the heart. Therefore, this study aims to translate US images with
spatially-varying resolution to ones with less spatially-varying resolution.
CycleGAN has been a prominent choice for unpaired medical image translation;
however, it neither guarantees structural consistency nor preserves
backscattering patterns between input and generated images for unpaired US
images. Approach. To circumvent this limitation, we propose a constrained
CycleGAN (CCycleGAN), which directly performs US image generation with unpaired
images acquired by different ultrasound array probes. In addition to
conventional adversarial and cycle-consistency losses of CycleGAN, CCycleGAN
introduces an identical loss and a correlation coefficient loss based on
intrinsic US backscattered signal properties to constrain structural
consistency and backscattering patterns, respectively. Instead of
post-processed B-mode images, CCycleGAN uses envelope data directly obtained
from beamformed radio-frequency signals without any other non-linear
postprocessing. Main Results. In vitro phantom results demonstrate that
CCycleGAN successfully generates images with improved spatial resolution as
well as higher peak signal-to-noise ratio (PSNR) and structural similarity
(SSIM) compared with benchmarks. Significance. CCycleGAN-generated US images of
the in vivo human beating heart further facilitate higher quality heart wall
motion estimation than benchmarks-generated ones, particularly in deep regions
- …