The Synthetic Image TEsting Framework (SITEF) for the evaluation of multi-spectral image segmentation algorithms

ABSTRACT One of the most challenging tasks in Remote Sensing at present is how to handle the huge amounts of image data acquired every day by the existing Earth Observation Satellites (EOS). An alternative approach to the standard per-pixel analysis of multi-spectral EOS images has evolved over the last decade. Instead of focusing on individual image pixels, the object-based image analysis approach consists of partitioning an image into meaningful image-objects. One of the reasons for the development of object-based methods has been the dramatic increase in commercially available high resolution digital remote sensing imagery, with spatial resolutions of 5.0 m and finer [1]. Also it has been recognised that the image pixel is not a "natural" element of an image scene. A common element of all object-based image analysis systems is the segmentation stage, where the image is partitioned in a number of objects (or segments), which is clearly a critical stage of the whole process. If the segmentation fails to identify as an object a given element present in the image, the subsequent stages will generally be unable to recognise or to classify this element. An evaluation of the abilities and limitations of the segmentation algorithms used is therefore an important aspect of any object based image analysis system. However, there is no established standard procedure for the evaluation of the segmentation results produced for EOS images The purpose of this work is to present the Synthetic Image TEsting Framework (SITEF), a tool to evaluate the performance of segmentation algorithms on multi-spectral images. The method is based on the production of synthetic images with the spectral characteristics of the image pixels extracted from a signature multi-spectral image The methodology used here is an evolution of the method described in REFERENCES [1] G.J. Hay, G. Castilla, M.A. Wulder, J.R. Ruiz, "An automated object-based approach for the multiscale image segmentation of forest scene

Robust Motion Segmentation from Pairwise Matches

In this paper we address a classification problem that has not been considered before, namely motion segmentation given pairwise matches only. Our contribution to this unexplored task is a novel formulation of motion segmentation as a two-step process. First, motion segmentation is performed on image pairs independently. Secondly, we combine independent pairwise segmentation results in a robust way into the final globally consistent segmentation. Our approach is inspired by the success of averaging methods. We demonstrate in simulated as well as in real experiments that our method is very effective in reducing the errors in the pairwise motion segmentation and can cope with large number of mismatches

Joint Learning of Intrinsic Images and Semantic Segmentation

Semantic segmentation of outdoor scenes is problematic when there are variations in imaging conditions. It is known that albedo (reflectance) is invariant to all kinds of illumination effects. Thus, using reflectance images for semantic segmentation task can be favorable. Additionally, not only segmentation may benefit from reflectance, but also segmentation may be useful for reflectance computation. Therefore, in this paper, the tasks of semantic segmentation and intrinsic image decomposition are considered as a combined process by exploring their mutual relationship in a joint fashion. To that end, we propose a supervised end-to-end CNN architecture to jointly learn intrinsic image decomposition and semantic segmentation. We analyze the gains of addressing those two problems jointly. Moreover, new cascade CNN architectures for intrinsic-for-segmentation and segmentation-for-intrinsic are proposed as single tasks. Furthermore, a dataset of 35K synthetic images of natural environments is created with corresponding albedo and shading (intrinsics), as well as semantic labels (segmentation) assigned to each object/scene. The experiments show that joint learning of intrinsic image decomposition and semantic segmentation is beneficial for both tasks for natural scenes. Dataset and models are available at: https://ivi.fnwi.uva.nl/cv/intrinsegComment: ECCV 201