NEFI: Network Extraction From Images

Networks and network-like structures are amongst the central building blocks of many technological and biological systems. Given a mathematical graph representation of a network, methods from graph theory enable a precise investigation of its properties. Software for the analysis of graphs is widely available and has been applied to graphs describing large scale networks such as social networks, protein-interaction networks, etc. In these applications, graph acquisition, i.e., the extraction of a mathematical graph from a network, is relatively simple. However, for many network-like structures, e.g. leaf venations, slime molds and mud cracks, data collection relies on images where graph extraction requires domain-specific solutions or even manual. Here we introduce Network Extraction From Images, NEFI, a software tool that automatically extracts accurate graphs from images of a wide range of networks originating in various domains. While there is previous work on graph extraction from images, theoretical results are fully accessible only to an expert audience and ready-to-use implementations for non-experts are rarely available or insufficiently documented. NEFI provides a novel platform allowing practitioners from many disciplines to easily extract graph representations from images by supplying flexible tools from image processing, computer vision and graph theory bundled in a convenient package. Thus, NEFI constitutes a scalable alternative to tedious and error-prone manual graph extraction and special purpose tools. We anticipate NEFI to enable the collection of larger datasets by reducing the time spent on graph extraction. The analysis of these new datasets may open up the possibility to gain new insights into the structure and function of various types of networks. NEFI is open source and available http://nefi.mpi-inf.mpg.de

Graph Refinement based Airway Extraction using Mean-Field Networks and Graph Neural Networks

Graph refinement, or the task of obtaining subgraphs of interest from over-complete graphs, can have many varied applications. In this work, we extract trees or collection of sub-trees from image data by, first deriving a graph-based representation of the volumetric data and then, posing the tree extraction as a graph refinement task. We present two methods to perform graph refinement. First, we use mean-field approximation (MFA) to approximate the posterior density over the subgraphs from which the optimal subgraph of interest can be estimated. Mean field networks (MFNs) are used for inference based on the interpretation that iterations of MFA can be seen as feed-forward operations in a neural network. This allows us to learn the model parameters using gradient descent. Second, we present a supervised learning approach using graph neural networks (GNNs) which can be seen as generalisations of MFNs. Subgraphs are obtained by training a GNN-based graph refinement model to directly predict edge probabilities. We discuss connections between the two classes of methods and compare them for the task of extracting airways from 3D, low-dose, chest CT data. We show that both the MFN and GNN models show significant improvement when compared to one baseline method, that is similar to a top performing method in the EXACT'09 Challenge, and a 3D U-Net based airway segmentation model, in detecting more branches with fewer false positives.Comment: Accepted for publication at Medical Image Analysis. 14 page

Extraction of Airways using Graph Neural Networks

We present extraction of tree structures, such as airways, from image data as a graph refinement task. To this end, we propose a graph auto-encoder model that uses an encoder based on graph neural networks (GNNs) to learn embeddings from input node features and a decoder to predict connections between nodes. Performance of the GNN model is compared with mean-field networks in their ability to extract airways from 3D chest CT scans.Comment: Extended Abstract submitted to MIDL, 2018. 3 page

Graph Convolutional Network Using Adaptive Neighborhood Laplacian Matrix for Hyperspectral Images with Application to Rice Seed Image Classification

Graph convolutional neural network architectures combine feature extraction and convolutional layers for hyperspectral image classification. An adaptive neighborhood aggregation method based on statistical variance integrating the spatial information along with the spectral signature of the pixels is proposed for improving graph convolutional network classification of hyperspectral images. The spatial-spectral information is integrated into the adjacency matrix and processed by a single-layer graph convolutional network. The algorithm employs an adaptive neighborhood selection criteria conditioned by the class it belongs to. Compared to fixed window-based feature extraction, this method proves effective in capturing the spectral and spatial features with variable pixel neighborhood sizes. The experimental results from the Indian Pines, Houston University, and Botswana Hyperion hyperspectral image datasets show that the proposed AN-GCN can significantly improve classification accuracy. For example, the overall accuracy for Houston University data increases from 81.71% (MiniGCN) to 97.88% (AN-GCN). Furthermore, the AN-GCN can classify hyperspectral images of rice seeds exposed to high day and night temperatures, proving its efficacy in discriminating the seeds under increased ambient temperature treatments

Automatic Foreground Initialization for Binary Image Segmentation

Foreground segmentation is a fundamental problem in computer vision. A popular approach for foreground extraction is through graph cuts in energy minimization framework. Most existing graph cuts based image segmentation algorithms rely on user’s initialization. In this work, we aim to find an automatic initialization for graph cuts. Unlike many previous methods, no additional training dataset is needed. Collecting a training set is not only expensive and time consuming, but it also may bias the algorithm to the particular data distribution of the collected dataset. We assume that the foreground differs significantly from the background in some unknown feature space and try to find the rectangle that is most different from the rest of the image by measuring histograms dissimilarity. We extract multiple features, design a ranking function to select good features, and compute histograms based on integral images. The standard graph cuts binary segmentation is applied, based on the color models learned from the initial rectangular segmentation. Then the steps of refining the color models and re-segmenting the image iterate in the grabcut manner, until convergence, which is guaranteed. The foreground detection algorithm performs well and the segmentation is further improved by graph cuts. We evaluate our method on three datasets with manually labelled foreground regions, and show that we reach the similar level of accuracy compared to previous work. Our approach, however, has an advantage over the previous work that we do not require a training dataset

Image segmentation in the wavelet domain using N-cut framework

We introduce a wavelet domain image segmentation algorithm based on Normalized Cut (NCut) framework in this thesis. By employing the NCut algorithm we solve the perceptual grouping problem of image segmentation which aims at the extraction of the global impression of an image. We capitalize on the reduced set of data to be processed and statistical features derived from the wavelet-transformed images to solve graph partitioning more efficiently than before. Five orientation histograms are computed to evaluate similarity/dissimilarity measure of local structure. We use properties of the wavelet transform filtering to capture edge information in vertical, horizontal and diagonal orientations. This approach allows for direct processing of compressed data and results in faster implementation of NCut framework than that in the spatial domain and also decent quality of segmentation of natural scene images