257 research outputs found
Learning Rigid Image Registration - Utilizing Convolutional Neural Networks for Medical Image Registration
Many traditional computer vision tasks, such as segmentation, have seen large step-changes in accuracy and/or speed with the application of Convolutional Neural Networks (CNNs). Image registration, the alignment of two or more images to a common space, is a fundamental step in many medical imaging workflows. In this paper we investigate whether these techniques can also bring tangible benefits to the registration task. We describe and evaluate the use of convolutional neural networks (CNNs) for both mono- and multi- modality registration and compare their performance to more traditional schemes, namely multi-scale, iterative registration. This paper also investigates incorporating inverse consistency of the learned spatial transformations to impose additional constraints on the network during training and investigate any benefit in accuracy during detection. The approaches are validated with a series of artificial mono-modal registration tasks utilizing T1-weighted MR brain i mages from the Open Access Series of Imaging Studies (OASIS) study and IXI brain development dataset and a series of real multi-modality registration tasks using T1-weighted and T2-weighted MR brain images from the 2015 Ischemia Stroke Lesion segmentation (ISLES) challenge. The results demonstrate that CNNs give excellent performance for both mono- and multi- modality head and neck registration compared to the baseline method with significantly fewer outliers and lower mean errors
TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning
Background and objective:
Processing of medical images such as MRI or CT presents different challenges compared to RGB images typically used in computer vision. These include a lack of labels for large datasets, high computational costs, and the need of metadata to describe the physical properties of voxels. Data augmentation is used to artificially increase the size of the training datasets. Training with image subvolumes or patches decreases the need for computational power. Spatial metadata needs to be carefully taken into account in order to ensure a correct alignment and orientation of volumes.
Methods:
We present TorchIO, an open-source Python library to enable efficient loading, preprocessing, augmentation and patch-based sampling of medical images for deep learning. TorchIO follows the style of PyTorch and integrates standard medical image processing libraries to efficiently process images during training of neural networks. TorchIO transforms can be easily composed, reproduced, traced and extended. Most transforms can be inverted, making the library suitable for test-time augmentation and estimation of aleatoric uncertainty in the context of segmentation. We provide multiple generic preprocessing and augmentation operations as well as simulation of MRI-specific artifacts.
Results:
Source code, comprehensive tutorials and extensive documentation for TorchIO can be found at http://torchio.rtfd.io/. The package can be installed from the Python Package Index (PyPI) running pip install torchio. It includes a command-line interface which allows users to apply transforms to image files without using Python. Additionally, we provide a graphical user interface within a TorchIO extension in 3D Slicer to visualize the effects of transforms.
Conclusion:
TorchIO was developed to help researchers standardize medical image processing pipelines and allow them to focus on the deep learning experiments. It encourages good open-science practices, as it supports experiment reproducibility and is version-controlled so that the software can be cited precisely. Due to its modularity, the library is compatible with other frameworks for deep learning with medical images
IBEX:A versatile multiplex optical imaging approach for deep phenotyping and spatial analysis of cells in complex tissues
The diverse composition of mammalian tissues poses challenges for understanding the cell–cell interactions required for organ homeostasis and how spatial relationships are perturbed during disease. Existing methods such as single-cell genomics, lacking a spatial context, and traditional immunofluorescence, capturing only two to six molecular features, cannot resolve these issues. Imaging technologies have been developed to address these problems, but each possesses limitations that constrain widespread use. Here we report a method that overcomes major impediments to highly multiplex tissue imaging. “Iterative bleaching extends multiplexity” (IBEX) uses an iterative staining and chemical bleaching method to enable high-resolution imaging of >65 parameters in the same tissue section without physical degradation. IBEX can be employed with various types of conventional microscopes and permits use of both commercially available and user-generated antibodies in an “open” system to allow easy adjustment of staining panels based on ongoing marker discovery efforts. We show how IBEX can also be used with amplified staining methods for imaging strongly fixed tissues with limited epitope retention and with oligonucleotide-based staining, allowing potential cross-referencing between flow cytometry, cellular indexing of transcriptomes and epitopes by sequencing, and IBEX analysis of the same tissue. To facilitate data processing, we provide an open-source platform for automated registration of iterative images. IBEX thus represents a technology that can be rapidly integrated into most current laboratory workflows to achieve high-content imaging to reveal the complex cellular landscape of diverse organs and tissues
NiftyNet: a deep-learning platform for medical imaging
Medical image analysis and computer-assisted intervention problems are
increasingly being addressed with deep-learning-based solutions. Established
deep-learning platforms are flexible but do not provide specific functionality
for medical image analysis and adapting them for this application requires
substantial implementation effort. Thus, there has been substantial duplication
of effort and incompatible infrastructure developed across many research
groups. This work presents the open-source NiftyNet platform for deep learning
in medical imaging. The ambition of NiftyNet is to accelerate and simplify the
development of these solutions, and to provide a common mechanism for
disseminating research outputs for the community to use, adapt and build upon.
NiftyNet provides a modular deep-learning pipeline for a range of medical
imaging applications including segmentation, regression, image generation and
representation learning applications. Components of the NiftyNet pipeline
including data loading, data augmentation, network architectures, loss
functions and evaluation metrics are tailored to, and take advantage of, the
idiosyncracies of medical image analysis and computer-assisted intervention.
NiftyNet is built on TensorFlow and supports TensorBoard visualization of 2D
and 3D images and computational graphs by default.
We present 3 illustrative medical image analysis applications built using
NiftyNet: (1) segmentation of multiple abdominal organs from computed
tomography; (2) image regression to predict computed tomography attenuation
maps from brain magnetic resonance images; and (3) generation of simulated
ultrasound images for specified anatomical poses.
NiftyNet enables researchers to rapidly develop and distribute deep learning
solutions for segmentation, regression, image generation and representation
learning applications, or extend the platform to new applications.Comment: Wenqi Li and Eli Gibson contributed equally to this work. M. Jorge
Cardoso and Tom Vercauteren contributed equally to this work. 26 pages, 6
figures; Update includes additional applications, updated author list and
formatting for journal submissio
PocketNet: A Smaller Neural Network for Medical Image Analysis
Medical imaging deep learning models are often large and complex, requiring
specialized hardware to train and evaluate these models. To address such
issues, we propose the PocketNet paradigm to reduce the size of deep learning
models by throttling the growth of the number of channels in convolutional
neural networks. We demonstrate that, for a range of segmentation and
classification tasks, PocketNet architectures produce results comparable to
that of conventional neural networks while reducing the number of parameters by
multiple orders of magnitude, using up to 90% less GPU memory, and speeding up
training times by up to 40%, thereby allowing such models to be trained and
deployed in resource-constrained settings
- …