Accurate Automatic Segmentation of Amygdala Subnuclei and Modeling of Uncertainty via Bayesian Fully Convolutional Neural Network

Recent advances in deep learning have improved the segmentation accuracy of subcortical brain structures, which would be useful in neuroimaging studies of many neurological disorders. However, most of the previous deep learning work does not investigate the specific difficulties that exist in segmenting extremely small but important brain regions such as the amygdala and its subregions. To tackle this challenging task, a novel 3D Bayesian fully convolutional neural network was developed to apply a dilated dualpathway approach that retains fine details and utilizes both local and more global contextual information to automatically segment the amygdala and its subregions at high precision. The proposed method provides insights on network design and sampling strategy that target segmentations of small 3D structures. In particular, this study confirms that a large context, enabled by a large field of view, is beneficial for segmenting small objects; furthermore, precise contextual information enabled by dilated convolutions allows for better boundary localization, which is critical for examining the morphology of the structure. In addition, it is demonstrated that the uncertainty information estimated from our network may be leveraged to identify atypicality in data. Our method was compared with two state-of-the-art deep learning models and a traditional multi-atlas approach, and exhibited excellent performance as measured both by Dice overlap as well as average symmetric surface distance. To the best of our knowledge, this work is the first deep learning-based approach that targets the subregions of the amygdala

Deep analytics of atomically-resolved images: manifest and latent features

Recent advances in scanning transmission electron and scanning tunneling microscopies allow researchers to measure materials structural and electronic properties, such as atomic displacements and charge density modulations, at an Angstrom scale in real space. At the same time, the ability to quickly acquire large, high-resolution datasets has created a challenge for rapid physics-based analysis of images that typically contain several hundreds to several thousand atomic units. Here we demonstrate a universal deep-learning based framework for locating and characterizing atomic species in the lattice, which can be applied to different types of atomically resolved measurements on different materials. Specifically, by inspecting and categorizing features in the output layer of a convolutional neural network, we are able to detect structural and electronic 'anomalies' associated with the presence of point defects in a tungsten disulfide monolayer, non-uniformity of the charge density distribution around specific lattice sites on the surface of strongly correlated oxides, and transition between different structural states of buckybowl molecules. We further extended our method towards tracking, from one image frame to another, minute distortions in the geometric shape of individual Si dumbbells in a 3-dimensional Si sample, which are associated with a motion of lattice defects and impurities. Due the applicability of our framework to both scanning tunneling microscopy and scanning transmission electron microscopy measurements, it can provide a fast and straightforward way towards creating a unified database of defect-property relationships from experimental data for each material

Multi-region segmentation of bladder cancer structures in MRI with progressive dilated convolutional networks

Precise segmentation of bladder walls and tumor regions is an essential step towards non-invasive identification of tumor stage and grade, which is critical for treatment decision and prognosis of patients with bladder cancer (BC). However, the automatic delineation of bladder walls and tumor in magnetic resonance images (MRI) is a challenging task, due to important bladder shape variations, strong intensity inhomogeneity in urine and very high variability across population, particularly on tumors appearance. To tackle these issues, we propose to use a deep fully convolutional neural network. The proposed network includes dilated convolutions to increase the receptive field without incurring extra cost nor degrading its performance. Furthermore, we introduce progressive dilations in each convolutional block, thereby enabling extensive receptive fields without the need for large dilation rates. The proposed network is evaluated on 3.0T T2-weighted MRI scans from 60 pathologically confirmed patients with BC. Experiments shows the proposed model to achieve high accuracy, with a mean Dice similarity coefficient of 0.98, 0.84 and 0.69 for inner wall, outer wall and tumor region, respectively. These results represent a very good agreement with reference contours and an increase in performance compared to existing methods. In addition, inference times are less than a second for a whole 3D volume, which is between 2-3 orders of magnitude faster than related state-of-the-art methods for this application. We showed that a CNN can yield precise segmentation of bladder walls and tumors in bladder cancer patients on MRI. The whole segmentation process is fully-automatic and yields results in very good agreement with the reference standard, demonstrating the viability of deep learning models for the automatic multi-region segmentation of bladder cancer MRI images.Comment: Published at the journal of Medical Physic

Densely Dilated Spatial Pooling Convolutional Network using benign loss functions for imbalanced volumetric prostate segmentation

The high incidence rate of prostate disease poses a requirement in early detection for diagnosis. As one of the main imaging methods used for prostate cancer detection, Magnetic Resonance Imaging (MRI) has wide range of appearance and imbalance problems, making automated prostate segmentation fundamental but challenging. Here we propose a novel Densely Dilated Spatial Pooling Convolutional Network (DDSP ConNet) in encoder-decoder structure. It employs dense structure to combine dilated convolution and global pooling, thus supplies coarse segmentation results from encoder and decoder subnet and preserves more contextual information. To obtain richer hierarchical feature maps, residual long connection is furtherly adopted to fuse contexture features. Meanwhile, we adopt DSC loss and Jaccard loss functions to train our DDSP ConNet. We surprisingly found and proved that, in contrast to re-weighted cross entropy, DSC loss and Jaccard loss have a lot of benign properties in theory, including symmetry, continuity and differentiability about the parameters of network. Extensive experiments on the MICCAI PROMISE12 challenge dataset have been done to corroborate the effectiveness of our DDSP ConNet with DSC loss and Jaccard loss. Totally, our method achieves a score of 85.78 in the test dataset, outperforming most of other competitors.Comment: 14pages, 5 figures, anonymous review in IJACAI201

Survey of Computer Vision and Machine Learning in Gastrointestinal Endoscopy

This paper attempts to provide the reader a place to begin studying the application of computer vision and machine learning to gastrointestinal (GI) endoscopy. They have been classified into 18 categories. It should be be noted by the reader that this is a review from pre-deep learning era. A lot of deep learning based applications have not been covered in this thesis

Instance Segmentation of Biological Images Using Harmonic Embeddings

We present a new instance segmentation approach tailored to biological images, where instances may correspond to individual cells, organisms or plant parts. Unlike instance segmentation for user photographs or road scenes, in biological data object instances may be particularly densely packed, the appearance variation may be particularly low, the processing power may be restricted, while, on the other hand, the variability of sizes of individual instances may be limited. The proposed approach successfully addresses these peculiarities. Our approach describes each object instance using an expectation of a limited number of sine waves with frequencies and phases adjusted to particular object sizes and densities. At train time, a fully-convolutional network is learned to predict the object embeddings at each pixel using a simple pixelwise regression loss, while at test time the instances are recovered using clustering in the embedding space. In the experiments, we show that our approach outperforms previous embedding-based instance segmentation approaches on a number of biological datasets, achieving state-of-the-art on a popular CVPPP benchmark. This excellent performance is combined with computational efficiency that is needed for deployment to domain specialists. The source code of the approach is available at https://github.com/kulikovv/harmonicComment: Accepted as oral to CVPR 202

DS-PASS: Detail-Sensitive Panoramic Annular Semantic Segmentation through SwaftNet for Surrounding Sensing

Semantically interpreting the traffic scene is crucial for autonomous transportation and robotics systems. However, state-of-the-art semantic segmentation pipelines are dominantly designed to work with pinhole cameras and train with narrow Field-of-View (FoV) images. In this sense, the perception capacity is severely limited to offer higher-level confidence for upstream navigation tasks. In this paper, we propose a network adaptation framework to achieve Panoramic Annular Semantic Segmentation (PASS), which allows to re-use conventional pinhole-view image datasets, enabling modern segmentation networks to comfortably adapt to panoramic images. Specifically, we adapt our proposed SwaftNet to enhance the sensitivity to details by implementing attention-based lateral connections between the detail-critical encoder layers and the context-critical decoder layers. We benchmark the performance of efficient segmenters on panoramic segmentation with our extended PASS dataset, demonstrating that the proposed real-time SwaftNet outperforms state-of-the-art efficient networks. Furthermore, we assess real-world performance when deploying the Detail-Sensitive PASS (DS-PASS) system on a mobile robot and an instrumented vehicle, as well as the benefit of panoramic semantics for visual odometry, showing the robustness and potential to support diverse navigational applications.Comment: 8 pages, 10 figure

Phase Segmentation in Atom-Probe Tomography Using Deep Learning-Based Edge Detection

Atom-probe tomography (APT) facilitates nano- and atomic-scale characterization and analysis of microstructural features. Specifically, APT is well suited to study the interfacial properties of granular or heterophase systems. Traditionally, the identification of the interface between, for precipitate and matrix phases, in APT data has been obtained either by extracting iso-concentration surfaces based on a user-supplied concentration value or by manually perturbing the concentration value until the iso-concentration surface qualitatively matches the interface. These approaches are subjective, not scalable, and may lead to inconsistencies due to local composition inhomogeneities. We propose a digital image segmentation approach based on deep neural networks that transfer learned knowledge from natural images to automatically segment the data obtained from APT into different phases. This approach not only provides an efficient way to segment the data and extract interfacial properties but does so without the need for expensive interface labeling for training the segmentation model. We consider here a system with a precipitate phase in a matrix and with three different interface modalities---layered, isolated, and interconnected---that are obtained for different relative geometries of the precipitate phase. We demonstrate the accuracy of our segmentation approach through qualitative visualization of the interfaces, as well as through quantitative comparisons with proximity histograms obtained by using more traditional approaches.Comment: 23 pages, 6 figure

Computer-Aided Knee Joint Magnetic Resonance Image Segmentation - A Survey

Osteoarthritis (OA) is one of the major health issues among the elderly population. MRI is the most popular technology to observe and evaluate the progress of OA course. However, the extreme labor cost of MRI analysis makes the process inefficient and expensive. Also, due to human error and subjective nature, the inter- and intra-observer variability is rather high. Computer-aided knee MRI segmentation is currently an active research field because it can alleviate doctors and radiologists from the time consuming and tedious job, and improve the diagnosis performance which has immense potential for both clinic and scientific research. In the past decades, researchers have investigated automatic/semi-automatic knee MRI segmentation methods extensively. However, to the best of our knowledge, there is no comprehensive survey paper in this field yet. In this survey paper, we classify the existing methods by their principles and discuss the current research status and point out the future research trend in-depth.Comment: 10 pages, 6 table

A Deep Learning Algorithm for One-step Contour Aware Nuclei Segmentation of Histopathological Images

This paper addresses the task of nuclei segmentation in high-resolution histopathological images. We propose an auto- matic end-to-end deep neural network algorithm for segmenta- tion of individual nuclei. A nucleus-boundary model is introduced to predict nuclei and their boundaries simultaneously using a fully convolutional neural network. Given a color normalized image, the model directly outputs an estimated nuclei map and a boundary map. A simple, fast and parameter-free post-processing procedure is performed on the estimated nuclei map to produce the final segmented nuclei. An overlapped patch extraction and assembling method is also designed for seamless prediction of nuclei in large whole-slide images. We also show the effectiveness of data augmentation methods for nuclei segmentation task. Our experiments showed our method outperforms prior state-of-the- art methods. Moreover, it is efficient that one 1000X1000 image can be segmented in less than 5 seconds. This makes it possible to precisely segment the whole-slide image in acceptable timeComment: 13 pages. 12 figure