Deep GrabCut for Object Selection

Most previous bounding-box-based segmentation methods assume the bounding box tightly covers the object of interest. However it is common that a rectangle input could be too large or too small. In this paper, we propose a novel segmentation approach that uses a rectangle as a soft constraint by transforming it into an Euclidean distance map. A convolutional encoder-decoder network is trained end-to-end by concatenating images with these distance maps as inputs and predicting the object masks as outputs. Our approach gets accurate segmentation results given sloppy rectangles while being general for both interactive segmentation and instance segmentation. We show our network extends to curve-based input without retraining. We further apply our network to instance-level semantic segmentation and resolve any overlap using a conditional random field. Experiments on benchmark datasets demonstrate the effectiveness of the proposed approaches.Comment: BMVC 201

Deep Extreme Cut: From Extreme Points to Object Segmentation

This paper explores the use of extreme points in an object (left-most, right-most, top, bottom pixels) as input to obtain precise object segmentation for images and videos. We do so by adding an extra channel to the image in the input of a convolutional neural network (CNN), which contains a Gaussian centered in each of the extreme points. The CNN learns to transform this information into a segmentation of an object that matches those extreme points. We demonstrate the usefulness of this approach for guided segmentation (grabcut-style), interactive segmentation, video object segmentation, and dense segmentation annotation. We show that we obtain the most precise results to date, also with less user input, in an extensive and varied selection of benchmarks and datasets. All our models and code are publicly available on http://www.vision.ee.ethz.ch/~cvlsegmentation/dextr/.Comment: CVPR 2018 camera ready. Project webpage and code: http://www.vision.ee.ethz.ch/~cvlsegmentation/dextr

Discrete-Continuous ADMM for Transductive Inference in Higher-Order MRFs

This paper introduces a novel algorithm for transductive inference in higher-order MRFs, where the unary energies are parameterized by a variable classifier. The considered task is posed as a joint optimization problem in the continuous classifier parameters and the discrete label variables. In contrast to prior approaches such as convex relaxations, we propose an advantageous decoupling of the objective function into discrete and continuous subproblems and a novel, efficient optimization method related to ADMM. This approach preserves integrality of the discrete label variables and guarantees global convergence to a critical point. We demonstrate the advantages of our approach in several experiments including video object segmentation on the DAVIS data set and interactive image segmentation

Development of a pulmonary imaging biomarker pipeline for phenotyping of chronic lung disease

We designed and generated pulmonary imaging biomarker pipelines to facilitate high-throughput research and point-of-care use in patients with chronic lung disease. Image processing modules and algorithm pipelines were embedded within a graphical user interface (based on the .NET framework) for pulmonary magnetic resonance imaging (MRI) and x-ray computed-tomography (CT) datasets. The software pipelines were generated using C++ and included: (1) inhale

Surface loss for medical image segmentation

Last decades have witnessed an unprecedented expansion of medical data in various largescale and complex systems. While achieving a lot of successes in many complex medical problems, there are still some challenges to deal with. Class imbalance is one of the common problems of medical image segmentation. It occurs mostly when there is a severely unequal class distribution, for instance, when the size of target foreground region is several orders of magnitude less that the background region size. In such problems, typical loss functions used for convolutional neural networks (CNN) segmentation fail to deliver good performances. Widely used losses,e.g., Dice or cross-entropy, are based on regional terms. They assume that all classes are equally distributed. Thus, they tend to favor the majority class and misclassify the target class. To address this issue, the main objective of this work is to build a boundary loss, a distance based measure on the space of contours and not regions. We argue that a boundary loss can mitigate the problems of regional losses via introducing a complementary distance-based information. Our loss is inspired by discrete (graph-based) optimization techniques for computing gradient flows of curve evolution. Following an integral approach for computing boundary variations, we express a non-symmetric L2 distance on the space of shapes as a regional integral, which avoids completely local differential computations. Our boundary loss is the sum of linear functions of the regional softmax probability outputs of the network. Therefore, it can easily be combined with standard regional losses and implemented with any existing deep network architecture for N-dimensional segmentation (N-D). Experiments were carried on three benchmark datasets corresponding to increasingly unbalanced segmentation problems: Multi modal brain tumor segmentation (BRATS17), the ischemic stroke lesion (ISLES) and white matter hyperintensities (WMH). Used in conjunction with the region-based generalized Dice loss (GDL), our boundary loss improves performance significantly compared to GDL alone, reaching up to 8% improvement in Dice score and 10% improvement in Hausdorff score. It also yielded a more stable learning process

画像の局所統計量に基づくフォーカルブラー領域分割

　被写体と焦点距離の関係によって生じるフォーカルブラーは画像撮影に伴う典型的な現象であり，画像のブラー情報を解析する技術はコンピュータビジョンの重要課題の一つである．フォーカルブラーからはシーンの相対的な奥行きや，撮影者の注目領域などシーンに関する有用な情報が得られる．フォーカルブラー領域分割はこれらの情報を解析し有効に利用するための技術であり，様々なアプリケーションの性能向上に寄与する．本論文では，フォーカルブラー領域分割手法の精度向上を目的として，(1)ブラー特徴推定の阻害要因に頑健なブラー特徴推定，(2)単一画像に対するブラー領域分割，および(3)2 枚の画像を用いたブラー領域分割手法を提案する．さらに，フォーカルブラー領域分割手法を含む2値領域分割の有効性を適切に評価するため，クラスタリングとクラス分類の文脈に基づいてブラー領域分割精度評価尺度を検証する．本論文ではブラー特徴推定の阻害要因に頑健なブラー特徴量としてANGHS (Amplitude-Normalized Gradient Histogram Span) を提案する．ANGHSは局所領域の輝度勾配を輝度振幅で正規化し，さらに輝度勾配ヒストグラムの裾の重さを評価する．本論文が提案するANGHSは局所領域内の輝度変化の少ない画素集合に対する頑健性に加え，輝度振幅に対する頑健性を備えている．単一画像に対するブラー領域分割では，ブラー特徴マップの識別性能が精度に大きく影響する点に着目し，識別性能の高いブラー特徴マップ推定法を提案する．ブラー特徴マップの識別性能向上のために，(i)複数サイズのグリッド分割を利用したスパースブラー特徴マップ推定と(ii)EAI (Edge Aware Interpolation) によるブラー特徴マップ推定を適用する．さらに領域分割ではまず，大津法を用いてブラー特徴マップを初期分割し，その後，初期分割結果と色特徴，およびブラー特徴を併用したGraphcutsを用いて初期分割結果を修正することで，ノンパラメトリック推定に基づく大域的領域分割とエネルギー最小化に基づく領域の高精細化によって精度を向上させる2段階領域分割を提案する．2枚の画像を用いたブラー領域分割手法では，2枚のブラーが異なる画像対からブラー差分特徴を求めることで，被写体と背景を分割する理論的なしきい値が定義できることに着目する．2枚のフォーカルブラー画像から推定したブラー差分特徴マップを理論的なしきい値で分割する．さらに，色特徴と被写体合焦画像から求めたブラー特徴マップを併用したGraphcutsで初期分割結果を補正することで精度の向上を図る．フォーカルブラー領域分割の精度評価では，2値領域分割がクラスタリングとクラス分類の問題として捉えられる点に着目し，各文脈における最適な評価尺度を検証する．本論文では，クラスタリングとクラス分類の各文脈についてフォーカルブラー領域分割精度評価のための要求事項を定義する．要求事項についてF1 Score, Intersection over Union, Accuracy, Matthews Correlation Coefficient, Informednessの各評価尺度を比較し，クラスタリングとクラス分類の各文脈において，Informednessの絶対値とInformednessがそれぞれ最適な評価尺度であることを示す．さらに，アルゴリズムを複数の観点から比較可能な統計的要約手法として，複数の領域分割パラメータを試行した際の最高精度と平均精度を用いた統計的要約手法を提案する．　精度評価では，ブラー特徴マップの識別性能評価，単一画像に対するブラー領域分割の精度評価，2枚の画像を用いたブラー領域分割の精度評価を行う．最初に，ブラー特徴マップの識別性能評価では5種類の従来手法によるブラー特徴マップと比較した結果，提案手法によるクラス分類の最高分割性能は0:780 ポイントの精度となり従来手法に対して最小で0:092ポイント，最大で0:366ポイント精度が向上した．また，大津法を用いた際のクラス分類における分割性能は0:697ポイントの精度となり，従来手法に対して最小で0:201ポイント，最大で0:400ポイント精度が向上した．次に，単一画像に対するブラー領域分割精度を比較した．提案領域分割手法は，従来手法を含むすべてのブラー特徴マップに対してクラス分類における分割精度が改善しており，汎用性の高い手法となっている。提案手法はクラス分類において0:722ポイントの精度となり，従来手法に対して最小で0:158ポイント，最大で0:373ポイント精度が向上した．最後に，2 枚の画像を用いたブラー領域分割の精度評価では，単一画像に対するブラー領域分割と精度比較を行った．2枚の画像を用いたブラー領域分割はシンプルな被写体で0:988ポイントの精度となり，単一画像に対する領域分割に対して0:095ポイント精度が向上した．複雑な花画像においては2枚の画像を用いたブラー領域分割は0:827ポイントの精度となり，単一画像に対する領域分割に対して0:058ポイント精度が向上した．また，単一画像に対するブラー領域分割では分割性能が悪い画像に対しても2枚の画像を用いたブラー領域分割は精度が改善されており，提案手法の有効性を示した．電気通信大学201

Optimization for Image Segmentation

Image segmentation, i.e., assigning each pixel a discrete label, is an essential task in computer vision with lots of applications. Major techniques for segmentation include for example Markov Random Field (MRF), Kernel Clustering (KC), and nowadays popular Convolutional Neural Networks (CNN). In this work, we focus on optimization for image segmentation. Techniques like MRF, KC, and CNN optimize MRF energies, KC criteria, or CNN losses respectively, and their corresponding optimization is very different. We are interested in the synergy and the complementary benefits of MRF, KC, and CNN for interactive segmentation and semantic segmentation. Our first contribution is pseudo-bound optimization for binary MRF energies that are high-order or non-submodular. Secondly, we propose Kernel Cut, a novel formulation for segmentation, which combines MRF regularization with Kernel Clustering. We show why to combine KC with MRF and how to optimize the joint objective. In the third part, we discuss how deep CNN segmentation can benefit from non-deep (i.e., shallow) methods like MRF and KC. In particular, we propose regularized losses for weakly-supervised CNN segmentation, in which we can integrate MRF energy or KC criteria as part of the losses. Minimization of regularized losses is a principled approach to semi-supervised learning, in general. Our regularized loss method is very simple and allows different kinds of regularization losses for CNN segmentation. We also study the optimization of regularized losses beyond gradient descent. Our regularized losses approach achieves state-of-the-art accuracy in semantic segmentation with near full supervision quality