Optimization for Image Segmentation

Image segmentation, i.e., assigning each pixel a discrete label, is an essential task in computer vision with lots of applications. Major techniques for segmentation include for example Markov Random Field (MRF), Kernel Clustering (KC), and nowadays popular Convolutional Neural Networks (CNN). In this work, we focus on optimization for image segmentation. Techniques like MRF, KC, and CNN optimize MRF energies, KC criteria, or CNN losses respectively, and their corresponding optimization is very different. We are interested in the synergy and the complementary benefits of MRF, KC, and CNN for interactive segmentation and semantic segmentation. Our first contribution is pseudo-bound optimization for binary MRF energies that are high-order or non-submodular. Secondly, we propose Kernel Cut, a novel formulation for segmentation, which combines MRF regularization with Kernel Clustering. We show why to combine KC with MRF and how to optimize the joint objective. In the third part, we discuss how deep CNN segmentation can benefit from non-deep (i.e., shallow) methods like MRF and KC. In particular, we propose regularized losses for weakly-supervised CNN segmentation, in which we can integrate MRF energy or KC criteria as part of the losses. Minimization of regularized losses is a principled approach to semi-supervised learning, in general. Our regularized loss method is very simple and allows different kinds of regularization losses for CNN segmentation. We also study the optimization of regularized losses beyond gradient descent. Our regularized losses approach achieves state-of-the-art accuracy in semantic segmentation with near full supervision quality

SegNema: Nematode segmentation strategy in digital microscopy images using deep learning and shape models

Proyecto de Graduación (Maestría en Computación con énfasis en Ciencias de la Computación) Instituto Tecnológico de Costa Rica, Escuela de Ingeniería en Computación, 2019.Nematodes are the most numerous multicellular animals on Earth and their study has a direct impact in the improvement and development of agricultural activities. This document introduces SegNema, a strategy for the segmentation of nematodes in microscopy images where deep learning is used for classification of pixels as nematode or background, and a shape model is used to associate landmarks that describe the position of the nematode in the image. To train the segmentation model, a set of 2939 manually labeled uncompressed images of size 1024 ⇥ 768 pixels obtained from 13 di↵erent sequences of microscopy images is used. The landmarks that describe the position of the nematodes in these training images are used to adjust a model capable of representing shapes corresponding to a nematode. The disparity between the shapes of the regions classified as nematode in the segmentation stage and their possible truncated representation with the shape model is used to rule out possible erroneous classifications. The validation of this model was performed on 321 images of the microscopy sequences that were not used in the training stage. In each image used for training and validation, there is information on the position of landmarks where a single nematode is delimited although more nematodes may be present.Los nematodos son los animales pluricelulares más numerosos en la Tierra y su estudio tiene un impacto en el desarrollo de actividades agrícolas. En este documento se introduce SegNema, una estrategia para la segmentación de nematodos en imágenes de microscopia donde se utiliza aprendizaje profundo para clasificación de píxeles como nematodo o fondo, y modelos de forma para asociar hitos que describen la posición del nematodo en la imagen. Para entrenar el modelo de segmentación se usan 2939 imágenes sin comprimir etiquetadas manualmente de tamaño 1024 ⇥ 768 p´ıxeles obtenidas de 13 secuencias de imágenes de microscopia. Por otro lado, los hitos que describen la posición de los nematodos en estas imágenes de entrenamiento son utilizados para ajustar un modelo capaz de representar formas correspondientes a nematodo. La disparidad entre formas de las regiones clasificadas como nematodo en la etapa de segmentación y su posible representación truncada con el modelo de forma es usado para descartar posibles clasificaciones err´oneas. Para la validación de este modelo se usan 321 imágenes de las secuencias de microscopia que no son utilizadas en la etapa de entrenamiento. En cada imagen usada para entrenamiento y validación existe la información de la posici´on de hitos donde se delimita un único nematodo aunque otros nematodos pueden estar presentes

Higher-order Losses and Optimization for Low-level and Deep Segmentation

Regularized objectives are common in low-level and deep segmentation. Regularization incorporates prior knowledge into objectives or losses. It represents constraints necessary to address ill-posedness, data noise, outliers, lack of supervision, etc. However, such constraints come at significant costs. First, regularization priors may lead to unintended biases, known or unknown. Since these can adversely affect specific applications, it is important to understand the causes & effects of these biases and to develop their solutions. Second, common regularized objectives are highly non-convex and present challenges for optimization. As known in low-level vision, first-order approaches like gradient descent are significantly weaker than more advanced algorithms. Yet, variants of the gradient descent dominate optimization of the loss functions for deep neural networks due to their size and complexity. Hence, standard segmentation networks still require an overwhelming amount of precise pixel-level supervision for training. This thesis addresses three related problems concerning higher-order objectives and higher-order optimizers. First, we focus on a challenging application—unsupervised vascular tree extraction in large 3D volumes containing complex ``entanglements" of near-capillary vessels. In the context of vasculature with unrestricted topology, we propose a new general curvature-regularizing model for arbitrarily complex one-dimensional curvilinear structures. In contrast, the standard surface regularization methods are impractical for thin vessels due to strong shrinking bias or the complexity of Gaussian/min curvature modeling for two-dimensional manifolds. In general, the shrinking bias is one well-known example of bias in the standard regularization methods. The second contribution of this thesis is a characterization of other new forms of biases in classical segmentation models that were not understood in the past. We develop new theories establishing data density biases in common pair-wise or graph-based clustering objectives, such as kernel K-means and normalized cut. This theoretical understanding inspires our new segmentation algorithms avoiding such biases. The third contribution of the thesis is a new optimization algorithm addressing the limitations of gradient descent in the context of regularized losses for deep learning. Our general trust-region algorithm can be seen as a high-order chain rule for network training. It can use many standard low-level regularizers and their powerful solvers. We improve the state-of-the-art in weakly-supervised semantic segmentation using a well-motivated low-level regularization model and its graph-cut solver

Advanced Computational Methods for Oncological Image Analysis

[Cancer is the second most common cause of death worldwide and encompasses highly variable clinical and biological scenarios. Some of the current clinical challenges are (i) early diagnosis of the disease and (ii) precision medicine, which allows for treatments targeted to specific clinical cases. The ultimate goal is to optimize the clinical workflow by combining accurate diagnosis with the most suitable therapies. Toward this, large-scale machine learning research can define associations among clinical, imaging, and multi-omics studies, making it possible to provide reliable diagnostic and prognostic biomarkers for precision oncology. Such reliable computer-assisted methods (i.e., artificial intelligence) together with clinicians’ unique knowledge can be used to properly handle typical issues in evaluation/quantification procedures (i.e., operator dependence and time-consuming tasks). These technical advances can significantly improve result repeatability in disease diagnosis and guide toward appropriate cancer care. Indeed, the need to apply machine learning and computational intelligence techniques has steadily increased to effectively perform image processing operations—such as segmentation, co-registration, classification, and dimensionality reduction—and multi-omics data integration.