Scale Object Selection (SOS) through a hierarchical segmentation by a multi-spectral per-pixel classification

International audienceIn high resolution multispectral optical data, the spatial detail of the images are generally smaller than the dimensions of objects, and often the spectral signature of pixels is not directly representative of classes we are interested in. Thus, taking into account the relations between groups of pixels becomes increasingly important, making object­oriented approaches preferable. In this work several scales of detail within an image are considered through a hierarchical segmentation approach, while the spectral information content of each pixel is accounted for by a per­pixel classification. The selection of the most suitable spatial scale for each class is obtained by merging the hierarchical segmentation and the per­pixel classification through the Scale Object Selection (SOS) algorithm. The SOS algorithm starts processing data from the highest level of the hierarchical segmentation, which has the least amount of spatial detail, down to the last segmentation map. At each segmentation level, objects are assigned to a specific class whenever the percentage of pixels belonging to the latter, according to a pixel­based procedure, exceeds a predefined threshold, thereby automatically selecting the most appropriate spatial scale for the classification of each object. We apply our method to multispectral, panchromatic and pan­sharpened QuickBird images

Knowledge-Based Classification of Grassland Ecosystem Based on Multi-Temporal WorldView-2 Data and FAO-LCCS Taxonomy

Grassland ecosystems can provide a variety of services for humans, such as carbon storage, food production, crop pollination and pest regulation. However, grasslands are today one of the most endangered ecosystems due to land use change, agricultural intensification, land abandonment as well as climate change. The present study explores the performance of a knowledge-driven GEOgraphic-Object—based Image Analysis (GEOBIA) learning scheme to classify Very High Resolution(VHR)imagesfornaturalgrasslandecosystemmapping. Theclassificationwasappliedto a Natura 2000 protected area in Southern Italy. The Food and Agricultural Organization Land Cover Classification System (FAO-LCCS) hierarchical scheme was instantiated in the learning phase of the algorithm. Four multi-temporal WorldView-2 (WV-2) images were classified by combining plant phenology and agricultural practices rules with prior-image spectral knowledge. Drawing on this knowledge, spectral bands and entropy features from one single date (Post Peak of Biomass) were firstly used for multiple-scale image segmentation into Small Objects (SO) and Large Objects (LO). Thereafter, SO were labelled by considering spectral and context-sensitive features from the whole multi-seasonal data set available together with ancillary data. Lastly, the labelled SO were overlaid to LO segments and, in turn, the latter were labelled by adopting FAO-LCCS criteria about the SOs presence dominance in each LO. Ground reference samples were used only for validating the SO and LO output maps. The knowledge driven GEOBIA classifier for SO classification obtained an OA value of 97.35% with an error of 0.04. For LO classification the value was 75.09% with an error of 0.70. At SO scale, grasslands ecosystem was classified with 92.6%, 99.9% and 96.1% of User’s, Producer’s Accuracy and F1-score, respectively. The findings reported indicate that the knowledge-driven approach not only can be applied for (semi)natural grasslands ecosystem mapping in vast and not accessible areas but can also reduce the costs of ground truth data acquisition. The approach used may provide different level of details (small and large objects in the scene) but also indicates how to design and validate local conservation policies

Remote Sensing Methods and Applications for Detecting Change in Forest Ecosystems

Forest ecosystems are being altered by climate change, invasive species, and additional stressors. Our ability to detect these changes and quantify their impacts relies on detailed data across spatial and temporal scales. This dissertation expands the ecological utility of long-term satellite imagery by developing high quality forest mapping products and examining spatiotemporal changes in tree species abundance and phenology across the northeastern United States (US; the ‘Northeast’). Species/genus-level forest composition maps were developed by integrating field data and Landsat images to model abundance at a sub-pixel scale. These abundance maps were then used to 1) produce a more detailed, accurate forest classification compared to similar products and 2) construct a 30-year time-series of abundance for eight common species/genera. Analyzing the time-series data revealed significant abundance trends in notable species, including increases in American beech (Fagus grandifolia) at the expense of sugar maple (Acer saccharum). Climate was the dominant predictor of abundance trends, indicating climate change may be altering competitive relationships. Spatiotemporal trends in deciduous forest phenology – start and end of the growing season (SOS/EOS) – were examined based on MODIS imagery from 2001-2015. SOS exhibited a slight advancing trend across the Northeast, but with a distinct spatial pattern: eastern ecoregions showed advance and western ecoregions delay. EOS trended substantially later almost everywhere. SOS trends were linked to winter-spring temperature and precipitation trends; areas with higher elevation and fall precipitation anomalies had negative associations with EOS trends. Together, this work demonstrates the value of remote sensing in furthering our understanding of long-term forest responses to changing environmental conditions. By highlighting potential changes in forest composition and function, the research presented here can be used to develop forest conservation and management strategies in the Northeast

What's wrong with classes? The theory of Knowledge

This paper wants to investigate the deepest meaning of the word class that is often used in machine learning and classification as a well-defined concept. This adventure will lead the reader to the fundamentals of Mathematics like set theory from Zermelo-Fraenkel. This will be our start, like is all Mathematics, to understand how well defined is the class concept. A broader theory will be outlined with the courageous attempt to give an homogenous framework to deal with machine learning problems

Visual saliency computation for image analysis

Visual saliency computation is about detecting and understanding salient regions and elements in a visual scene. Algorithms for visual saliency computation can give clues to where people will look in images, what objects are visually prominent in a scene, etc. Such algorithms could be useful in a wide range of applications in computer vision and graphics. In this thesis, we study the following visual saliency computation problems. 1) Eye Fixation Prediction. Eye fixation prediction aims to predict where people look in a visual scene. For this problem, we propose a Boolean Map Saliency (BMS) model which leverages the global surroundedness cue using a Boolean map representation. We draw a theoretic connection between BMS and the Minimum Barrier Distance (MBD) transform to provide insight into our algorithm. Experiment results show that BMS compares favorably with state-of-the-art methods on seven benchmark datasets. 2) Salient Region Detection. Salient region detection entails computing a saliency map that highlights the regions of dominant objects in a scene. We propose a salient region detection method based on the Minimum Barrier Distance (MBD) transform. We present a fast approximate MBD transform algorithm with an error bound analysis. Powered by this fast MBD transform algorithm, our method can run at about 80 FPS and achieve state-of-the-art performance on four benchmark datasets. 3) Salient Object Detection. Salient object detection targets at localizing each salient object instance in an image. We propose a method using a Convolutional Neural Network (CNN) model for proposal generation and a novel subset optimization formulation for bounding box filtering. In experiments, our subset optimization formulation consistently outperforms heuristic bounding box filtering baselines, such as Non-maximum Suppression, and our method substantially outperforms previous methods on three challenging datasets. 4) Salient Object Subitizing. We propose a new visual saliency computation task, called Salient Object Subitizing, which is to predict the existence and the number of salient objects in an image using holistic cues. To this end, we present an image dataset of about 14K everyday images which are annotated using an online crowdsourcing marketplace. We show that an end-to-end trained CNN subitizing model can achieve promising performance without requiring any localization process. A method is proposed to further improve the training of the CNN subitizing model by leveraging synthetic images. 5) Top-down Saliency Detection. Unlike the aforementioned tasks, top-down saliency detection entails generating task-specific saliency maps. We propose a weakly supervised top-down saliency detection approach by modeling the top-down attention of a CNN image classifier. We propose Excitation Backprop and the concept of contrastive attention to generate highly discriminative top-down saliency maps. Our top-down saliency detection method achieves superior performance in weakly supervised localization tasks on challenging datasets. The usefulness of our method is further validated in the text-to-region association task, where our method provides state-of-the-art performance using only weakly labeled web images for training

ISPRS International Journal of Geo-Information / Extraction of terraces on the loess plateau from high-resolution DEMs and imagery utilizing object-based image analysis

Terraces are typical artificial landforms on the Loess Plateau, with ecological functions in water and soil conservation, agricultural production, and biodiversity. Recording the spatial distribution of terraces is the basis of monitoring their extent and understanding their ecological effects. The current terrace extraction method mainly relies on high-resolution imagery, but its accuracy is limited due to vegetation coverage distorting the features of terraces in imagery. High-resolution topographic data reflecting the morphology of true terrace surfaces are needed. Terraces extraction on the Loess Plateau is challenging because of the complex terrain and diverse vegetation after the implementation of “vegetation recovery”. This study presents an automatic method of extracting terraces based on 1 m resolution digital elevation models (DEMs) and 0.3 m resolution Worldview-3 imagery as auxiliary information used for object-based image analysis (OBIA). A multi-resolution segmentation method was used where slope, positive and negative terrain index (PN), accumulative curvature slope (AC), and slope of slope (SOS) were determined as input layers for image segmentation by correlation analysis and Sheffield entropy method. The main classification features based on DEMs were chosen from the terrain features derived from terrain factors and texture features by gray-level co-occurrence matrix (GLCM) analysis; subsequently, these features were determined by the importance analysis on classification and regression tree (CART) analysis. Extraction rules based on DEMs were generated from the classification features with a total classification accuracy of 89.96%. The red band and near-infrared band of images were used to exclude construction land, which is easily confused with small-size terraces. As a result, the total classification accuracy was increased to 94%. The proposed method ensures comprehensive consideration of terrain, texture, shape, and spectrum characteristics, demonstrating huge potential in hilly-gully loess region with similarly complex terrain and diverse vegetation covers.(VLID)219512

LiDAR and Orthophoto Synergy to optimize Object-Based Landscape Change:Analysis of an Active Landslide

Active landslides have three major effects on landscapes: (1) land cover change, (2) topographical change, and (3) above ground biomass change. Data derived from multi-temporal Light Detection and Ranging technology (LiDAR) are used in combination with multi-temporal orthophotos to quantify these changes between 2006 and 2012, caused by an active deep-seated landslide near the village of Doren in Austria. Land-cover is classified by applying membership-based classification and contextual improvements based on the synergy of orthophotos and LiDAR-based elevation data. Topographical change is calculated by differencing of LiDAR derived digital terrain models. The above ground biomass is quantified by applying a local-maximum algorithm for tree top detection, in combination with allometric equations. The land cover classification accuracies were improved from 65% (using only LiDAR) and 76% (using only orthophotos) to 90% (using data synergy) for 2006. A similar increase from respectively 64% and 75% to 91% was established for 2012. The increased accuracies demonstrate the effectiveness of using data synergy of LiDAR and orthophotos using object-based image analysis to quantify landscape changes, caused by an active landslide. The method has great potential to be transferred to larger areas for use in landscape change analyses

Towards a 20m global building map from Sentinel-1 SAR Data

This study introduces a technique for automatically mapping built-up areas using synthetic aperture radar (SAR) backscattering intensity and interferometric multi-temporal coherence generated from Sentinel-1 data in the framework of the Copernicus program. The underlying hypothesis is that, in SAR images, built-up areas exhibit very high backscattering values that are coherent in time. Several particular characteristics of the Sentinel-1 satellite mission are put to good use, such as its high revisit time, the availability of dual-polarized data, and its small orbital tube. The newly developed algorithm is based on an adaptive parametric thresholding that first identifies pixels with high backscattering values in both VV and VH polarimetric channels. The interferometric SAR coherence is then used to reduce false alarms. These are caused by land cover classes (other than buildings) that are characterized by high backscattering values that are not coherent in time (e.g., certain types of vegetated areas). The algorithm was tested on Sentinel-1 Interferometric Wide Swath data from five different test sites located in semiarid and arid regions in the Mediterranean region and Northern Africa. The resulting building maps were compared with the Global Urban Footprint (GUF) derived from the TerraSAR-X mission data and, on average, a 92% agreement was obtained.Peer ReviewedPostprint (published version

Computational Methods for Segmentation of Multi-Modal Multi-Dimensional Cardiac Images

Segmentation of the heart structures helps compute the cardiac contractile function quantified via the systolic and diastolic volumes, ejection fraction, and myocardial mass, representing a reliable diagnostic value. Similarly, quantification of the myocardial mechanics throughout the cardiac cycle, analysis of the activation patterns in the heart via electrocardiography (ECG) signals, serve as good cardiac diagnosis indicators. Furthermore, high quality anatomical models of the heart can be used in planning and guidance of minimally invasive interventions under the assistance of image guidance. The most crucial step for the above mentioned applications is to segment the ventricles and myocardium from the acquired cardiac image data. Although the manual delineation of the heart structures is deemed as the gold-standard approach, it requires significant time and effort, and is highly susceptible to inter- and intra-observer variability. These limitations suggest a need for fast, robust, and accurate semi- or fully-automatic segmentation algorithms. However, the complex motion and anatomy of the heart, indistinct borders due to blood flow, the presence of trabeculations, intensity inhomogeneity, and various other imaging artifacts, makes the segmentation task challenging. In this work, we present and evaluate segmentation algorithms for multi-modal, multi-dimensional cardiac image datasets. Firstly, we segment the left ventricle (LV) blood-pool from a tri-plane 2D+time trans-esophageal (TEE) ultrasound acquisition using local phase based filtering and graph-cut technique, propagate the segmentation throughout the cardiac cycle using non-rigid registration-based motion extraction, and reconstruct the 3D LV geometry. Secondly, we segment the LV blood-pool and myocardium from an open-source 4D cardiac cine Magnetic Resonance Imaging (MRI) dataset by incorporating average atlas based shape constraint into the graph-cut framework and iterative segmentation refinement. The developed fast and robust framework is further extended to perform right ventricle (RV) blood-pool segmentation from a different open-source 4D cardiac cine MRI dataset. Next, we employ convolutional neural network based multi-task learning framework to segment the myocardium and regress its area, simultaneously, and show that segmentation based computation of the myocardial area is significantly better than that regressed directly from the network, while also being more interpretable. Finally, we impose a weak shape constraint via multi-task learning framework in a fully convolutional network and show improved segmentation performance for LV, RV and myocardium across healthy and pathological cases, as well as, in the challenging apical and basal slices in two open-source 4D cardiac cine MRI datasets. We demonstrate the accuracy and robustness of the proposed segmentation methods by comparing the obtained results against the provided gold-standard manual segmentations, as well as with other competing segmentation methods

Salient Object Detection Techniques in Computer Vision-A Survey.

Detection and localization of regions of images that attract immediate human visual attention is currently an intensive area of research in computer vision. The capability of automatic identification and segmentation of such salient image regions has immediate consequences for applications in the field of computer vision, computer graphics, and multimedia. A large number of salient object detection (SOD) methods have been devised to effectively mimic the capability of the human visual system to detect the salient regions in images. These methods can be broadly categorized into two categories based on their feature engineering mechanism: conventional or deep learning-based. In this survey, most of the influential advances in image-based SOD from both conventional as well as deep learning-based categories have been reviewed in detail. Relevant saliency modeling trends with key issues, core techniques, and the scope for future research work have been discussed in the context of difficulties often faced in salient object detection. Results are presented for various challenging cases for some large-scale public datasets. Different metrics considered for assessment of the performance of state-of-the-art salient object detection models are also covered. Some future directions for SOD are presented towards end