207 research outputs found
Simulating Patho-realistic Ultrasound Images using Deep Generative Networks with Adversarial Learning
Ultrasound imaging makes use of backscattering of waves during their
interaction with scatterers present in biological tissues. Simulation of
synthetic ultrasound images is a challenging problem on account of inability to
accurately model various factors of which some include intra-/inter scanline
interference, transducer to surface coupling, artifacts on transducer elements,
inhomogeneous shadowing and nonlinear attenuation. Current approaches typically
solve wave space equations making them computationally expensive and slow to
operate. We propose a generative adversarial network (GAN) inspired approach
for fast simulation of patho-realistic ultrasound images. We apply the
framework to intravascular ultrasound (IVUS) simulation. A stage 0 simulation
performed using pseudo B-mode ultrasound image simulator yields speckle mapping
of a digitally defined phantom. The stage I GAN subsequently refines them to
preserve tissue specific speckle intensities. The stage II GAN further refines
them to generate high resolution images with patho-realistic speckle profiles.
We evaluate patho-realism of simulated images with a visual Turing test
indicating an equivocal confusion in discriminating simulated from real. We
also quantify the shift in tissue specific intensity distributions of the real
and simulated images to prove their similarity.Comment: To appear in the Proceedings of the 2018 IEEE International Symposium
on Biomedical Imaging (ISBI 2018
Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks
Sonography synthesis has a wide range of applications, including medical
procedure simulation, clinical training and multimodality image registration.
In this paper, we propose a machine learning approach to simulate ultrasound
images at given 3D spatial locations (relative to the patient anatomy), based
on conditional generative adversarial networks (GANs). In particular, we
introduce a novel neural network architecture that can sample anatomically
accurate images conditionally on spatial position of the (real or mock)
freehand ultrasound probe. To ensure an effective and efficient spatial
information assimilation, the proposed spatially-conditioned GANs take
calibrated pixel coordinates in global physical space as conditioning input,
and utilise residual network units and shortcuts of conditioning data in the
GANs' discriminator and generator, respectively. Using optically tracked B-mode
ultrasound images, acquired by an experienced sonographer on a fetus phantom,
we demonstrate the feasibility of the proposed method by two sets of
quantitative results: distances were calculated between corresponding
anatomical landmarks identified in the held-out ultrasound images and the
simulated data at the same locations unseen to the networks; a usability study
was carried out to distinguish the simulated data from the real images. In
summary, we present what we believe are state-of-the-art visually realistic
ultrasound images, simulated by the proposed GAN architecture that is stable to
train and capable of generating plausibly diverse image samples.Comment: Accepted to MICCAI RAMBO 201
NiftyNet: a deep-learning platform for medical imaging
Medical image analysis and computer-assisted intervention problems are
increasingly being addressed with deep-learning-based solutions. Established
deep-learning platforms are flexible but do not provide specific functionality
for medical image analysis and adapting them for this application requires
substantial implementation effort. Thus, there has been substantial duplication
of effort and incompatible infrastructure developed across many research
groups. This work presents the open-source NiftyNet platform for deep learning
in medical imaging. The ambition of NiftyNet is to accelerate and simplify the
development of these solutions, and to provide a common mechanism for
disseminating research outputs for the community to use, adapt and build upon.
NiftyNet provides a modular deep-learning pipeline for a range of medical
imaging applications including segmentation, regression, image generation and
representation learning applications. Components of the NiftyNet pipeline
including data loading, data augmentation, network architectures, loss
functions and evaluation metrics are tailored to, and take advantage of, the
idiosyncracies of medical image analysis and computer-assisted intervention.
NiftyNet is built on TensorFlow and supports TensorBoard visualization of 2D
and 3D images and computational graphs by default.
We present 3 illustrative medical image analysis applications built using
NiftyNet: (1) segmentation of multiple abdominal organs from computed
tomography; (2) image regression to predict computed tomography attenuation
maps from brain magnetic resonance images; and (3) generation of simulated
ultrasound images for specified anatomical poses.
NiftyNet enables researchers to rapidly develop and distribute deep learning
solutions for segmentation, regression, image generation and representation
learning applications, or extend the platform to new applications.Comment: Wenqi Li and Eli Gibson contributed equally to this work. M. Jorge
Cardoso and Tom Vercauteren contributed equally to this work. 26 pages, 6
figures; Update includes additional applications, updated author list and
formatting for journal submissio
Endoscopic Ultrasound Image Synthesis Using a Cycle-Consistent Adversarial Network
Endoscopic ultrasound (EUS) is a challenging procedure that requires skill, both in endoscopy and ultrasound image interpretation. Classification of key anatomical landmarks visible on EUS images can assist the gastroenterologist during navigation. Current applications of deep learning have shown the ability to automatically classify ultrasound images with high accuracy. However, these techniques require a large amount of labelled data which is time consuming to obtain, and in the case of EUS, is also a difficult task to perform retrospectively due to the lack of 3D context. In this paper, we propose the use of an image-to-image translation method to create synthetic EUS (sEUS) images from CT data, that can be used as a data augmentation strategy when EUS data is scarce. We train a cycle-consistent adversarial network with unpaired EUS images and CT slices extracted in a manner such that they mimic plausible EUS views, to generate sEUS images from the pancreas, aorta and liver. We quantitatively evaluate the use of sEUS images in a classification sub-task and assess the Fréchet Inception Distance. We show that synthetic data, obtained from CT data, imposes only a minor classification accuracy penalty and may help generalization to new unseen patients. The code and a dataset containing generated sEUS images are available at: https://ebonmati.github.io
OnUVS: Online Feature Decoupling Framework for High-Fidelity Ultrasound Video Synthesis
Ultrasound (US) imaging is indispensable in clinical practice. To diagnose
certain diseases, sonographers must observe corresponding dynamic anatomic
structures to gather comprehensive information. However, the limited
availability of specific US video cases causes teaching difficulties in
identifying corresponding diseases, which potentially impacts the detection
rate of such cases. The synthesis of US videos may represent a promising
solution to this issue. Nevertheless, it is challenging to accurately animate
the intricate motion of dynamic anatomic structures while preserving image
fidelity. To address this, we present a novel online feature-decoupling
framework called OnUVS for high-fidelity US video synthesis. Our highlights can
be summarized by four aspects. First, we introduced anatomic information into
keypoint learning through a weakly-supervised training strategy, resulting in
improved preservation of anatomical integrity and motion while minimizing the
labeling burden. Second, to better preserve the integrity and textural
information of US images, we implemented a dual-decoder that decouples the
content and textural features in the generator. Third, we adopted a
multiple-feature discriminator to extract a comprehensive range of visual cues,
thereby enhancing the sharpness and fine details of the generated videos.
Fourth, we constrained the motion trajectories of keypoints during online
learning to enhance the fluidity of generated videos. Our validation and user
studies on in-house echocardiographic and pelvic floor US videos showed that
OnUVS synthesizes US videos with high fidelity.Comment: 14 pages, 13 figures and 6 table
Trustworthy and Intelligent COVID-19 Diagnostic IoMT through XR and Deep-Learning-Based Clinic Data Access
This article presents a novel extended reality (XR) and deep-learning-based Internet-of-Medical-Things (IoMT) solution for the COVID-19 telemedicine diagnostic, which systematically combines virtual reality/augmented reality (AR) remote surgical plan/rehearse hardware, customized 5G cloud computing and deep learning algorithms to provide real-time COVID-19 treatment scheme clues. Compared to existing perception therapy techniques, our new technique can significantly improve performance and security. The system collected 25 clinic data from the 347 positive and 2270 negative COVID-19 patients in the Red Zone by 5G transmission. After that, a novel auxiliary classifier generative adversarial network-based intelligent prediction algorithm is conducted to train the new COVID-19 prediction model. Furthermore, The Copycat network is employed for the model stealing and attack for the IoMT to improve the security performance. To simplify the user interface and achieve an excellent user experience, we combined the Red Zone's guiding images with the Green Zone's view through the AR navigate clue by using 5G. The XR surgical plan/rehearse framework is designed, including all COVID-19 surgical requisite details that were developed with a real-time response guaranteed. The accuracy, recall, F1-score, and area under the ROC curve (AUC) area of our new IoMT were 0.92, 0.98, 0.95, and 0.98, respectively, which outperforms the existing perception techniques with significantly higher accuracy performance. The model stealing also has excellent performance, with the AUC area of 0.90 in Copycat slightly lower than the original model. This study suggests a new framework in the COVID-19 diagnostic integration and opens the new research about the integration of XR and deep learning for IoMT implementation
- …