Recurrent Saliency Transformation Network: Incorporating Multi-Stage Visual Cues for Small Organ Segmentation

We aim at segmenting small organs (e.g., the pancreas) from abdominal CT scans. As the target often occupies a relatively small region in the input image, deep neural networks can be easily confused by the complex and variable background. To alleviate this, researchers proposed a coarse-to-fine approach, which used prediction from the first (coarse) stage to indicate a smaller input region for the second (fine) stage. Despite its effectiveness, this algorithm dealt with two stages individually, which lacked optimizing a global energy function, and limited its ability to incorporate multi-stage visual cues. Missing contextual information led to unsatisfying convergence in iterations, and that the fine stage sometimes produced even lower segmentation accuracy than the coarse stage. This paper presents a Recurrent Saliency Transformation Network. The key innovation is a saliency transformation module, which repeatedly converts the segmentation probability map from the previous iteration as spatial weights and applies these weights to the current iteration. This brings us two-fold benefits. In training, it allows joint optimization over the deep networks dealing with different input scales. In testing, it propagates multi-stage visual information throughout iterations to improve segmentation accuracy. Experiments in the NIH pancreas segmentation dataset demonstrate the state-of-the-art accuracy, which outperforms the previous best by an average of over 2%. Much higher accuracies are also reported on several small organs in a larger dataset collected by ourselves. In addition, our approach enjoys better convergence properties, making it more efficient and reliable in practice.Comment: Accepted to CVPR 2018 (10 pages, 6 figures

Deep grey matter volumetry as a function of age using a semi-automatic qMRI algorithm

Quantitative Magnetic Resonance has become more and more accepted for clinical trial in many fields. This technique not only can generate qMRI maps (such as T1/T2/PD) but also can be used for further postprocessing including segmentation of brain and characterization of different brain tissue. Another main application of qMRI is to measure the volume of the brain tissue such as the deep Grey Matter (dGM). The deep grey matter serves as the brain's "relay station" which receives and sends inputs between the cortical brain regions. An abnormal volume of the dGM is associated with certain diseases such as Fetal Alcohol Spectrum Disorders (FASD). The goal of this study is to investigate the effect of age on the volume change of the dGM using qMRI. Thirteen patients (mean age= 26.7 years old and age range from 0.5 to 72.5 years old) underwent imaging at a 1.5T MR scanner. Axial images of the entire brain were acquired with the mixed Turbo Spin-echo (mixed -TSE) pulse sequence. The acquired mixed-TSE images were transferred in DICOM format image for further analysis using the MathCAD 2001i software (Mathsoft, Cambridge, MA). Quantitative T1 and T2-weighted MR images were generated. The image data sets were further segmented using the dual-space clustering segmentation. Then volume of the dGM matter was calculated using a pixel counting algorithm and the spectrum of the T1/T2/PD distribution were also generated. Afterwards, the dGM volume of each patient was calculated and plotted on scatter plot. The mean volume of the dGM, standard deviation, and range were also calculated. The result shows that volume of the dGM is 47.5 ±5.3ml (N=13) which is consistent with former studies. The polynomial tendency line generated based on scatter plot shows that the volume of the dGM gradually increases with age at early age and reaches the maximum volume around the age of 20, and then it starts to decrease gradually in adulthood and drops much faster in elderly age. This result may help scientists to understand more about the aging of the brain and it can also be used to compare with the results from former studies using different techniques

Computational Anatomy for Multi-Organ Analysis in Medical Imaging: A Review

The medical image analysis field has traditionally been focused on the development of organ-, and disease-specific methods. Recently, the interest in the development of more 20 comprehensive computational anatomical models has grown, leading to the creation of multi-organ models. Multi-organ approaches, unlike traditional organ-specific strategies, incorporate inter-organ relations into the model, thus leading to a more accurate representation of the complex human anatomy. Inter-organ relations are not only spatial, but also functional and physiological. Over the years, the strategies 25 proposed to efficiently model multi-organ structures have evolved from the simple global modeling, to more sophisticated approaches such as sequential, hierarchical, or machine learning-based models. In this paper, we present a review of the state of the art on multi-organ analysis and associated computation anatomy methodology. The manuscript follows a methodology-based classification of the different techniques 30 available for the analysis of multi-organs and multi-anatomical structures, from techniques using point distribution models to the most recent deep learning-based approaches. With more than 300 papers included in this review, we reflect on the trends and challenges of the field of computational anatomy, the particularities of each anatomical region, and the potential of multi-organ analysis to increase the impact of 35 medical imaging applications on the future of healthcare.Comment: Paper under revie

Computational approaches for the multimodal imaging of nanomaterials and their biochemical effects

Nanomaterial delivery systems constitute a group of drug delivery vehicles that have been used extensively in biodelivery. The proper characterization of the therapeutic function of these nanomaterials requires analytical methods to track the presence of the cargo and its biochemical effects. In some cases, the detection of the cargo and biochemical changes are not attainable in the same experiment, and more than one technique might be needed for the proper analysis of the drug delivery system. In this case, separate analysis of adjacent tissue sections is performed by techniques that offer complementary information such as MALDI-MS and LA-ICP-MS. However, the approaches to combine the information from these techniques to obtain insights into the mechanism of action of the nanomaterials have been limited to visual inspection and image overlay, which can only provide qualitative information. To advance towards a more quantitative analysis, in this dissertation we have developed computational techniques for image reconstruction, segmentation, and registration of MALDI-MS and LA-ICP-MS images to monitor the biodistribution, excretion and biochemical effects of nanomaterial delivery systems. First, we developed an open-source computational approach for LA-ICP-MS image reconstruction and segmentation using Python, which revealed that nanomaterials distribute in different sub-organ regions based on their chemical and physical properties. For instance, in the analysis of gold nanoparticles and bismuth nanorods, we find that the nanomaterials distribute in different regions of the spleen, suggesting differences in their biochemical interactions within the same organ. Next, we developed a computational workflow in Python to register LA-ICP-MS and MALDI-MS images using image registration approaches, obtaining a method with errors below 50 µm. Finally, we used the developed approaches for registration of LA-ICP-MS and MALDI-MS images to evaluate the delivery vehicles and cargo, obtaining quantitative information about the correlation of the signals obtained in the two image modalities. The use of image registration for the analysis of siRNA delivery via nanoparticle stabilized capsules (NPSC) reveals that expected changes in lipid levels occur at different locations than the NPSC accumulate, thus providing deeper insight into how siRNA delivery by NPSCs influences lipid biochemistry in vivo

Computational processing and analysis of ear images

Tese de mestrado. Engenharia Biomédica. Faculdade de Engenharia. Universidade do Porto. 201

Multiscale, multimodal analysis of tumor heterogeneity in IDH1 mutant vs wild-type diffuse gliomas.

Glioma is recognized to be a highly heterogeneous CNS malignancy, whose diverse cellular composition and cellular interactions have not been well characterized. To gain new clinical- and biological-insights into the genetically-bifurcated IDH1 mutant (mt) vs wildtype (wt) forms of glioma, we integrated data from protein, genomic and MR imaging from 20 treatment-naïve glioma cases and 16 recurrent GBM cases. Multiplexed immunofluorescence (MxIF) was used to generate single cell data for 43 protein markers representing all cancer hallmarks, Genomic sequencing (exome and RNA (normal and tumor) and magnetic resonance imaging (MRI) quantitative features (protocols were T1-post, FLAIR and ADC) from whole tumor, peritumoral edema and enhancing core vs equivalent normal region were also collected from patients. Based on MxIF analysis, 85,767 cells (glioma cases) and 56,304 cells (GBM cases) were used to generate cell-level data for 24 biomarkers. K-means clustering was used to generate 7 distinct groups of cells with divergent biomarker profiles and deconvolution was used to assign RNA data into three classes. Spatial and molecular heterogeneity metrics were generated for the cell data. All features were compared between IDH mt and IDHwt patients and were finally combined to provide a holistic/integrated comparison. Protein expression by hallmark was generally lower in the IDHmt vs wt patients. Molecular and spatial heterogeneity scores for angiogenesis and cell invasion also differed between IDHmt and wt gliomas irrespective of prior treatment and tumor grade; these differences also persisted in the MR imaging features of peritumoral edema and contrast enhancement volumes. A coherent picture of enhanced angiogenesis in IDHwt tumors was derived from multiple platforms (genomic, proteomic and imaging) and scales from individual proteins to cell clusters and heterogeneity, as well as bulk tumor RNA and imaging features. Longer overall survival for IDH1mt glioma patients may reflect mutation-driven alterations in cellular, molecular, and spatial heterogeneity which manifest in discernable radiological manifestations