Joint segmentation of color and depth data based on splitting and merging driven by surface fitting

This paper proposes a segmentation scheme based on the joint usage of color and depth data together with a 3D surface estimation scheme. Firstly a set of multi-dimensional vectors is built from color, geometry and surface orientation information. Normalized cuts spectral clustering is then applied in order to recursively segment the scene in two parts thus obtaining an over-segmentation. This procedure is followed by a recursive merging stage where close segments belonging to the same object are joined together. At each step of both procedures a NURBS model is fitted on the computed segments and the accuracy of the fitting is used as a measure of the plausibility that a segment represents a single surface or object. By comparing the accuracy to the one at the previous step, it is possible to determine if each splitting or merging operation leads to a better scene representation and consequently whether to perform it or not. Experimental results show how the proposed method provides an accurate and reliable segmentation

Geodesic Distance Histogram Feature for Video Segmentation

This paper proposes a geodesic-distance-based feature that encodes global information for improved video segmentation algorithms. The feature is a joint histogram of intensity and geodesic distances, where the geodesic distances are computed as the shortest paths between superpixels via their boundaries. We also incorporate adaptive voting weights and spatial pyramid configurations to include spatial information into the geodesic histogram feature and show that this further improves results. The feature is generic and can be used as part of various algorithms. In experiments, we test the geodesic histogram feature by incorporating it into two existing video segmentation frameworks. This leads to significantly better performance in 3D video segmentation benchmarks on two datasets

Segmentation and semantic labelling of RGBD data with convolutional neural networks and surface fitting

We present an approach for segmentation and semantic labelling of RGBD data exploiting together geometrical cues and deep learning techniques. An initial over-segmentation is performed using spectral clustering and a set of non-uniform rational B-spline surfaces is fitted on the extracted segments. Then a convolutional neural network (CNN) receives in input colour and geometry data together with surface fitting parameters. The network is made of nine convolutional stages followed by a softmax classifier and produces a vector of descriptors for each sample. In the next step, an iterative merging algorithm recombines the output of the over-segmentation into larger regions matching the various elements of the scene. The couples of adjacent segments with higher similarity according to the CNN features are candidate to be merged and the surface fitting accuracy is used to detect which couples of segments belong to the same surface. Finally, a set of labelled segments is obtained by combining the segmentation output with the descriptors from the CNN. Experimental results show how the proposed approach outperforms state-of-the-art methods and provides an accurate segmentation and labelling

Pilot investigation of remote sensing for intertidal oyster mapping in coastal South Carolina: a methods comparison

South Carolina’s oyster reefs are a major component of the coastal landscape. Eastern oysters Crassostrea virginica are an important economic resource to the state and serve many essential functions in the environment, including water filtration, creek bank stabilization and habitat for other plants and animals. Effective conservation and management of oyster reefs is dependent on an understanding of their abundance, distribution, condition, and change over time. In South Carolina, over 95% of the state’s oyster habitat is intertidal. The current intertidal oyster reef database for South Carolina was developed by field assessment over several years. This database was completed in the early 1980s and is in need of an update to assess resource/habitat status and trends across the state. Anthropogenic factors such as coastal development and associated waterway usage (e.g., boat wakes) are suspected of significantly altering the extent and health of the state’s oyster resources. In 2002 the NOAA Coastal Services Center’s (Center) Coastal Remote Sensing Program (CRS) worked with the Marine Resources Division of the South Carolina Department of Natural Resources (SCDNR) to develop methods for mapping intertidal oyster reefs along the South Carolina coast using remote sensing technology. The objective of this project was to provide SCDNR with potential methodologies and approaches for assessing oyster resources in a more efficiently than could be accomplished through field digitizing. The project focused on the utility of high-resolution aerial imagery and on documenting the effectiveness of various analysis techniques for accomplishing the update. (PDF contains 32 pages

Combining Contrast Invariant L1 Data Fidelities with Nonlinear Spectral Image Decomposition

This paper focuses on multi-scale approaches for variational methods and corresponding gradient flows. Recently, for convex regularization functionals such as total variation, new theory and algorithms for nonlinear eigenvalue problems via nonlinear spectral decompositions have been developed. Those methods open new directions for advanced image filtering. However, for an effective use in image segmentation and shape decomposition, a clear interpretation of the spectral response regarding size and intensity scales is needed but lacking in current approaches. In this context, $L^1$ data fidelities are particularly helpful due to their interesting multi-scale properties such as contrast invariance. Hence, the novelty of this work is the combination of $L^1$-based multi-scale methods with nonlinear spectral decompositions. We compare $L^1$ with $L^2$ scale-space methods in view of spectral image representation and decomposition. We show that the contrast invariant multi-scale behavior of $L^1-TV$ promotes sparsity in the spectral response providing more informative decompositions. We provide a numerical method and analyze synthetic and biomedical images at which decomposition leads to improved segmentation.Comment: 13 pages, 7 figures, conference SSVM 201