Recurrent Pixel Embedding for Instance Grouping

We introduce a differentiable, end-to-end trainable framework for solving pixel-level grouping problems such as instance segmentation consisting of two novel components. First, we regress pixels into a hyper-spherical embedding space so that pixels from the same group have high cosine similarity while those from different groups have similarity below a specified margin. We analyze the choice of embedding dimension and margin, relating them to theoretical results on the problem of distributing points uniformly on the sphere. Second, to group instances, we utilize a variant of mean-shift clustering, implemented as a recurrent neural network parameterized by kernel bandwidth. This recurrent grouping module is differentiable, enjoys convergent dynamics and probabilistic interpretability. Backpropagating the group-weighted loss through this module allows learning to focus on only correcting embedding errors that won't be resolved during subsequent clustering. Our framework, while conceptually simple and theoretically abundant, is also practically effective and computationally efficient. We demonstrate substantial improvements over state-of-the-art instance segmentation for object proposal generation, as well as demonstrating the benefits of grouping loss for classification tasks such as boundary detection and semantic segmentation

Automated Discrimination of Pathological Regions in Tissue Images: Unsupervised Clustering vs Supervised SVM Classification

Recognizing and isolating cancerous cells from non pathological tissue areas (e.g. connective stroma) is crucial for fast and objective immunohistochemical analysis of tissue images. This operation allows the further application of fully-automated techniques for quantitative evaluation of protein activity, since it avoids the necessity of a preventive manual selection of the representative pathological areas in the image, as well as of taking pictures only in the pure-cancerous portions of the tissue. In this paper we present a fully-automated method based on unsupervised clustering that performs tissue segmentations highly comparable with those provided by a skilled operator, achieving on average an accuracy of 90%. Experimental results on a heterogeneous dataset of immunohistochemical lung cancer tissue images demonstrate that our proposed unsupervised approach overcomes the accuracy of a theoretically superior supervised method such as Support Vector Machine (SVM) by 8%

Shape-driven segmentation of the arterial wall in intravascular ultrasound images

Segmentation of arterial wall boundaries from intravascular images is an important problem for many applications in the study of plaque characteristics, mechanical properties of the arterial wall, its 3D reconstruction, and its measurements such as lumen size, lumen radius, and wall radius. We present a shape-driven approach to segmentation of the arterial wall from intravascular ultrasound images in the rectangular domain. In a properly built shape space using training data, we constrain the lumen and media-adventitia contours to a smooth, closed geometry, which increases the segmentation quality without any tradeoff with a regularizer term. In addition to a shape prior, we utilize an intensity prior through a non-parametric probability density based image energy, with global image measurements rather than pointwise measurements used in previous methods. Furthermore, a detection step is included to address the challenges introduced to the segmentation process by side branches and calcifications. All these features greatly enhance our segmentation method. The tests of our algorithm on a large dataset demonstrate the effectiveness of our approach

Shape-driven segmentation of the arterial wall in intravascular ultrasound images

Segmentation of arterial wall boundaries from intravascular images is an important problem for many applications in the study of plaque characteristics, mechanical properties of the arterial wall, its 3D reconstruction, and its measurements such as lumen size, lumen radius, and wall radius. We present a shape-driven approach to segmentation of the arterial wall from intravascular ultrasound images in the rectangular domain. In a properly built shape space using training data, we constrain the lumen and media-adventitia contours to a smooth, closed geometry, which increases the segmentation quality without any tradeoff with a regularizer term. In addition to a shape prior, we utilize an intensity prior through a non-parametric probability density based image energy, with global image measurements rather than pointwise measurements used in previous methods. Furthermore, a detection step is included to address the challenges introduced to the segmentation process by side branches and calcifications. All these features greatly enhance our segmentation method. The tests of our algorithm on a large dataset demonstrate the effectiveness of our approach

A Projected Gradient Descent Method for CRF Inference allowing End-To-End Training of Arbitrary Pairwise Potentials

Are we using the right potential functions in the Conditional Random Field models that are popular in the Vision community? Semantic segmentation and other pixel-level labelling tasks have made significant progress recently due to the deep learning paradigm. However, most state-of-the-art structured prediction methods also include a random field model with a hand-crafted Gaussian potential to model spatial priors, label consistencies and feature-based image conditioning. In this paper, we challenge this view by developing a new inference and learning framework which can learn pairwise CRF potentials restricted only by their dependence on the image pixel values and the size of the support. Both standard spatial and high-dimensional bilateral kernels are considered. Our framework is based on the observation that CRF inference can be achieved via projected gradient descent and consequently, can easily be integrated in deep neural networks to allow for end-to-end training. It is empirically demonstrated that such learned potentials can improve segmentation accuracy and that certain label class interactions are indeed better modelled by a non-Gaussian potential. In addition, we compare our inference method to the commonly used mean-field algorithm. Our framework is evaluated on several public benchmarks for semantic segmentation with improved performance compared to previous state-of-the-art CNN+CRF models.Comment: Presented at EMMCVPR 2017 conferenc

Feasibility of automated 3-dimensional magnetic resonance imaging pancreas segmentation.

PurposeWith the advent of MR guided radiotherapy, internal organ motion can be imaged simultaneously during treatment. In this study, we evaluate the feasibility of pancreas MRI segmentation using state-of-the-art segmentation methods.Methods and materialT2 weighted HASTE and T1 weighted VIBE images were acquired on 3 patients and 2 healthy volunteers for a total of 12 imaging volumes. A novel dictionary learning (DL) method was used to segment the pancreas and compared to t mean-shift merging (MSM), distance regularized level set (DRLS), graph cuts (GC) and the segmentation results were compared to manual contours using Dice's index (DI), Hausdorff distance and shift of the-center-of-the-organ (SHIFT).ResultsAll VIBE images were successfully segmented by at least one of the auto-segmentation method with DI >0.83 and SHIFT ≤2 mm using the best automated segmentation method. The automated segmentation error of HASTE images was significantly greater. DL is statistically superior to the other methods in Dice's overlapping index. For the Hausdorff distance and SHIFT measurement, DRLS and DL performed slightly superior to the GC method, and substantially superior to MSM. DL required least human supervision and was faster to compute.ConclusionOur study demonstrated potential feasibility of automated segmentation of the pancreas on MRI images with minimal human supervision at the beginning of imaging acquisition. The achieved accuracy is promising for organ localization