Informative sample generation using class aware generative adversarial networks for classification of chest Xrays

Training robust deep learning (DL) systems for disease detection from medical images is challenging due to limited images covering different disease types and severity. The problem is especially acute, where there is a severe class imbalance. We propose an active learning (AL) framework to select most informative samples for training our model using a Bayesian neural network. Informative samples are then used within a novel class aware generative adversarial network (CAGAN) to generate realistic chest xray images for data augmentation by transferring characteristics from one class label to another. Experiments show our proposed AL framework is able to achieve state-of-the-art performance by using about $35\%$ of the full dataset, thus saving significant time and effort over conventional methods

3D Convolutional Neural Networks for Tumor Segmentation using Long-range 2D Context

We present an efficient deep learning approach for the challenging task of tumor segmentation in multisequence MR images. In recent years, Convolutional Neural Networks (CNN) have achieved state-of-the-art performances in a large variety of recognition tasks in medical imaging. Because of the considerable computational cost of CNNs, large volumes such as MRI are typically processed by subvolumes, for instance slices (axial, coronal, sagittal) or small 3D patches. In this paper we introduce a CNN-based model which efficiently combines the advantages of the short-range 3D context and the long-range 2D context. To overcome the limitations of specific choices of neural network architectures, we also propose to merge outputs of several cascaded 2D-3D models by a voxelwise voting strategy. Furthermore, we propose a network architecture in which the different MR sequences are processed by separate subnetworks in order to be more robust to the problem of missing MR sequences. Finally, a simple and efficient algorithm for training large CNN models is introduced. We evaluate our method on the public benchmark of the BRATS 2017 challenge on the task of multiclass segmentation of malignant brain tumors. Our method achieves good performances and produces accurate segmentations with median Dice scores of 0.918 (whole tumor), 0.883 (tumor core) and 0.854 (enhancing core). Our approach can be naturally applied to various tasks involving segmentation of lesions or organs.Comment: Submitted to the journal Computerized Medical Imaging and Graphic

An Information Theoretic Approach For Feature Selection And Segmentation In Posterior Fossa Tumors

Posterior Fossa (PF) is a type of brain tumor located in or near brain stem and cerebellum. About 55% - 70 % pediatric brain tumors arise in the posterior fossa, compared with only 15% - 20% of adult tumors. For segmenting PF tumors we should have features to study the characteristics of tumors. In literature, different types of texture features such as Fractal Dimension (FD) and Multifractional Brownian Motion (mBm) have been exploited for measuring randomness associated with brain and tumor tissues structures, and the varying appearance of tissues in magnetic resonance images (MRI). For selecting best features techniques such as neural network and boosting methods have been exploited. However, neural network cannot descirbe about the properties of texture features. We explore methods such as information theroetic methods which can perform feature selection based on properties of texture features. The primary contribution of this dissertation is investigating efficacy of different image features such as intensity, fractal texture, and level - set shape in segmentation of PF tumor for pediatric patients. We explore effectiveness of using four different feature selection and three different segmentation techniques respectively to discriminate tumor regions from normal tissue in multimodal brain MRI. Our research suggest that Kullback - Leibler Divergence (KLD) measure for feature ranking and selection and Expectation Maximization (EM) algorithm for feature fusion and tumor segmentation offer the best performance for the patient data in this study. To improve segmentation accuracy, we need to consider abnormalities such as cyst, edema and necrosis which surround tumors. In this work, we exploit features which describe properties of cyst and technique which can be used to segment it. To achieve this goal, we extend the two class KLD techniques to multiclass feature selection techniques, so that we can effectively select features for tumor, cyst and non tumor tissues. We compute segemntation accuracy by computing number of pixels segemented to total number of pixels for the best features. For automated process we integrate the inhomoheneity correction, feature selection using KLD and segmentation in an integrated EM framework. To validate results we have used similarity coefficients for computing the robustness of segmented tumor and cyst

Image similarity metrics suitable for infrared video stabilization during active wildfire monitoring : a comparative analysis

Aerial Thermal Infrared (TIR) imagery has demonstrated tremendous potential to monitor active forest fires and acquire detailed information about fire behavior. However, aerial video is usually unstable and requires inter-frame registration before further processing. Measurement of image misalignment is an essential operation for video stabilization. Misalignment can usually be estimated through image similarity, although image similarity metrics are also sensitive to other factors such as changes in the scene and lighting conditions. Therefore, this article presents a thorough analysis of image similarity measurement techniques useful for inter-frame registration in wildfire thermal video. Image similarity metrics most commonly and successfully employed in other fields were surveyed, adapted, benchmarked and compared. We investigated their response to different camera movement components as well as recording frequency and natural variations in fire, background and ambient conditions. The study was conducted in real video from six fire experimental scenarios, ranging from laboratory tests to large-scale controlled burns. Both Global and Local Sensitivity Analyses (GSA and LSA, respectively) were performed using state-of-the-art techniques. Based on the obtained results, two different similarity metrics are proposed to satisfy two different needs. A normalized version of Mutual Information is recommended as cost function during registration, whereas 2D correlation performed the best as quality control metric after registration. These results provide a sound basis for image alignment measurement and open the door to further developments in image registration, motion estimation and video stabilization for aerial monitoring of active wildland fires

Image similarity metrics suitable for infrared video stabilization during active wildfire monitoring: a comparative analysis

Aerial Thermal Infrared (TIR) imagery has demonstrated tremendous potential to monitor active forest fires and acquire detailed information about fire behavior. However, aerial video is usually unstable and requires inter-frame registration before further processing. Measurement of image misalignment is an essential operation for video stabilization. Misalignment can usually be estimated through image similarity, although image similarity metrics are also sensitive to other factors such as changes in the scene and lighting conditions. Therefore, this article presents a thorough analysis of image similarity measurement techniques useful for inter-frame registration in wildfire thermal video. Image similarity metrics most commonly and successfully employed in other fields were surveyed, adapted, benchmarked and compared. We investigated their response to different camera movement components as well as recording frequency and natural variations in fire, background and ambient conditions. The study was conducted in real video from six fire experimental scenarios, ranging from laboratory tests to large-scale controlled burns. Both Global and Local Sensitivity Analyses (GSA and LSA, respectively) were performed using state-of-the-art techniques. Based on the obtained results, two different similarity metrics are proposed to satisfy two different needs. A normalized version of Mutual Information is recommended as cost function during registration, whereas 2D correlation performed the best as quality control metric after registration.Peer ReviewedPostprint (published version

An Information-Theory Framework for Multi-Modal Visualization

The main goal of this master thesis is the development of new fusion strategies that enhance multimodal visualization strategies

Affine Registration of label maps in Label Space

Two key aspects of coupled multi-object shape\ud analysis and atlas generation are the choice of representation\ud and subsequent registration methods used to align the sample\ud set. For example, a typical brain image can be labeled into\ud three structures: grey matter, white matter and cerebrospinal\ud fluid. Many manipulations such as interpolation, transformation,\ud smoothing, or registration need to be performed on these images\ud before they can be used in further analysis. Current techniques\ud for such analysis tend to trade off performance between the two\ud tasks, performing well for one task but developing problems when\ud used for the other.\ud This article proposes to use a representation that is both\ud flexible and well suited for both tasks. We propose to map object\ud labels to vertices of a regular simplex, e.g. the unit interval for\ud two labels, a triangle for three labels, a tetrahedron for four\ud labels, etc. This representation, which is routinely used in fuzzy\ud classification, is ideally suited for representing and registering\ud multiple shapes. On closer examination, this representation\ud reveals several desirable properties: algebraic operations may\ud be done directly, label uncertainty is expressed as a weighted\ud mixture of labels (probabilistic interpretation), interpolation is\ud unbiased toward any label or the background, and registration\ud may be performed directly.\ud We demonstrate these properties by using label space in a gradient\ud descent based registration scheme to obtain a probabilistic\ud atlas. While straightforward, this iterative method is very slow,\ud could get stuck in local minima, and depends heavily on the initial\ud conditions. To address these issues, two fast methods are proposed\ud which serve as coarse registration schemes following which the\ud iterative descent method can be used to refine the results. Further,\ud we derive an analytical formulation for direct computation of the\ud "group mean" from the parameters of pairwise registration of all\ud the images in the sample set. We show results on richly labeled\ud 2D and 3D data sets

Deformable Image Registration for Hyperspectral Images

Image registration is one of the basic image processing operations in remote sensing. A hyperspectral image has two spatial dimensions and one spectral dimension. There are many hyperspectral sensors used in remote sensing. Traditional intensity-based registration methods may fail for hyperspectral images because of the different spectral sensitivities for different sensors. In addition, not all spectral bands are required to achieve accurate registration. This thesis develops a modification of the large deformation diffeomorphic metric mappings (LDDMM) algorithm in order to deal with the challenges when applied to hyperspectral images. The transformation generated by our method that deforms one image to match the other is differentiable, isomorphic and invertible. We also propose a mutual information based band selection algorithm to reduce the data redundancy of the hyperspectral images. The approach is applied to two hyperspectral images from OMEGA instrument, with a better matching result than original LDDMM method with respect to mutual information

Multi-Atlas based Segmentation of Multi-Modal Brain Images

Brain image analysis is playing a fundamental role in clinical and population-based epidemiological studies. Several brain disorder studies involve quantitative interpretation of brain scans and particularly require accurate measurement and delineation of tissue volumes in the scans. Automatic segmentation methods have been proposed to provide reliability and accuracy of the labelling as well as performing an automated procedure. Taking advantage of prior information about the brain's anatomy provided by an atlas as a reference model can help simplify the labelling process. The segmentation in the atlas-based approach will be problematic if the atlas and the target image are not accurately aligned, or if the atlas does not appropriately represent the anatomical structure/region. The accuracy of the segmentation can be improved by utilising a group of atlases. Employing multiple atlases brings about considerable issues in segmenting a new subject's brain image. Registering multiple atlases to the target scan and fusing labels from registered atlases, for a population obtained from different modalities, are challenging tasks: image-intensity comparisons may no longer be valid, since image brightness can have highly diff ering meanings in dfferent modalities. The focus is on the problem of multi-modality and methods are designed and developed to deal with this issue specifically in image registration and label fusion. To deal with multi-modal image registration, two independent approaches are followed. First, a similarity measure is proposed based upon comparing the self-similarity of each of the images to be aligned. Second, two methods are proposed to reduce the multi-modal problem to a mono-modal one by constructing representations not relying on the image intensities. Structural representations work on the basis of using un-decimated complex wavelet representation in one method, and modified approach using entropy in the other one. To handle the cross-modality label fusion, a method is proposed to weight atlases based on atlas-target similarity. The atlas-target similarity is measured by scale-based comparison taking advantage of structural features captured from un-decimated complex wavelet coefficients. The proposed methods are assessed using the simulated and real brain data from computed tomography images and different modes of magnetic resonance images. Experimental results reflect the superiority of the proposed methods over the classical and state-of-the art methods

Segmentation of brain MRI during early childhood

The objective of this thesis is the development of automatic methods to measure the changes in volume and growth of brain structures in prematurely born infants. Automatic tools for accurate tissue quantification from magnetic resonance images can provide means for understanding how the neurodevelopmental effects of the premature birth, such as cognitive, neurological or behavioural impairment, are related to underlying changes in brain anatomy. Understanding these changes forms a basis for development of suitable treatments to improve the outcomes of premature birth. In this thesis we focus on the segmentation of brain structures from magnetic resonance images during early childhood. Most of the current brain segmentation techniques have been focused on the segmentation of adult or neonatal brains. As a result of rapid development, the brain anatomy during early childhood differs from anatomy of both adult and neonatal brains and therefore requires adaptations of available techniques to produce good results. To address the issue of anatomical differences of the brain during early childhood compared to other age-groups, population-specific deformable and probabilistic atlases are introduced. A method for generation of population-specific prior information in form of a probabilistic atlas is proposed and used to enhance existing segmentation algorithms. The evaluation of registration-based and intensity-based approaches shows the techniques to be complementary in the quality of automatic segmentation in different parts of the brain. We propose a novel robust segmentation method combining the advantages of both approaches. The method is based on multiple label propagation using B-spline non-rigid registration followed by EM segmentation. Intensity inhomogeneity is a shading artefact resulting from the acquisition process, which significantly affects modern high resolution MR data acquired at higher magnetic field strengths. A novel template based method focused on correcting the intensity inhomogeneity in data acquired at higher magnetic field strengths is therefore proposed. The proposed segmentation method combined with proposed intensity inhomogeneity correction method offers a robust tool for quantification of volumes and growth of brain structures during early childhood. The tool have been applied to 67 T1-weigted images of subject at one and two years of age