Segmentation and Classification of Multimodal Imagery

Segmentation and classification are two important computer vision tasks that transform input data into a compact representation that allow fast and efficient analysis. Several challenges exist in generating accurate segmentation or classification results. In a video, for example, objects often change the appearance and are partially occluded, making it difficult to delineate the object from its surroundings. This thesis proposes video segmentation and aerial image classification algorithms to address some of the problems and provide accurate results. We developed a gradient driven three-dimensional segmentation technique that partitions a video into spatiotemporal objects. The algorithm utilizes the local gradient computed at each pixel location together with the global boundary map acquired through deep learning methods to generate initial pixel groups by traversing from low to high gradient regions. A local clustering method is then employed to refine these initial pixel groups. The refined sub-volumes in the homogeneous regions of video are selected as initial seeds and iteratively combined with adjacent groups based on intensity similarities. The volume growth is terminated at the color boundaries of the video. The over-segments obtained from the above steps are then merged hierarchically by a multivariate approach yielding a final segmentation map for each frame. In addition, we also implemented a streaming version of the above algorithm that requires a lower computational memory. The results illustrate that our proposed methodology compares favorably well, on a qualitative and quantitative level, in segmentation quality and computational efficiency with the latest state of the art techniques. We also developed a convolutional neural network (CNN)-based method to efficiently combine information from multisensor remotely sensed images for pixel-wise semantic classification. The CNN features obtained from multiple spectral bands are fused at the initial layers of deep neural networks as opposed to final layers. The early fusion architecture has fewer parameters and thereby reduces the computational time and GPU memory during training and inference. We also introduce a composite architecture that fuses features throughout the network. The methods were validated on four different datasets: ISPRS Potsdam, Vaihingen, IEEE Zeebruges, and Sentinel-1, Sentinel-2 dataset. For the Sentinel-1,-2 datasets, we obtain the ground truth labels for three classes from OpenStreetMap. Results on all the images show early fusion, specifically after layer three of the network, achieves results similar to or better than a decision level fusion mechanism. The performance of the proposed architecture is also on par with the state-of-the-art results

Best Merge Region Growing with Integrated Probabilistic Classification for Hyperspectral Imagery

A new method for spectral-spatial classification of hyperspectral images is proposed. The method is based on the integration of probabilistic classification within the hierarchical best merge region growing algorithm. For this purpose, preliminary probabilistic support vector machines classification is performed. Then, hierarchical step-wise optimization algorithm is applied, by iteratively merging regions with the smallest Dissimilarity Criterion (DC). The main novelty of this method consists in defining a DC between regions as a function of region statistical and geometrical features along with classification probabilities. Experimental results are presented on a 200-band AVIRIS image of the Northwestern Indiana s vegetation area and compared with those obtained by recently proposed spectral-spatial classification techniques. The proposed method improves classification accuracies when compared to other classification approaches

Integrating spatial and spectral information for automatic feature identification in high -resolution remotely sensed images

This research used image objects, instead of pixels, as the basic unit of analysis in high-resolution imagery. Thus, not only spectral radiance and texture were used in the analysis, but also spatial context. Furthermore, the automated identification of attributed objects is potentially useful for integrating remote sensing with a vector-based GIS.;A study area in Morgantown, WV was chosen as a site for the development and testing of automated feature extraction methods with high-resolution data. In the first stage of the analysis, edges were identified using texture. Experiments with simulated data indicated that a linear operator identified curved and sharp edges more accurately than square shaped operators. Areas with edges that formed a closed boundary were used to delineate sub-patches. In the region growing step, the similarities of all adjacent subpatches were examined using a multivariate Hotelling T2 test that draws on the classes\u27 covariance matrices. Sub-patches that were not sufficiently dissimilar were merged to form image patches.;Patches were then classified into seven classes: Building, Road, Forest, Lawn, Shadowed Vegetation, Water, and Shadow. Six classification methods were compared: the pixel-based ISODATA and maximum likelihood approaches, field-based ECHO, and region based maximum likelihood using patch means, a divergence index, and patch probability density functions (pdfs). Classification with the divergence index showed the lowest accuracy, a kappa index of 0.254. The highest accuracy, 0.783, was obtained from classification using the patch pdf. This classification also produced a visually pleasing product, with well-delineated objects and without the distracting salt-and-pepper effect of isolated misclassified pixels. The accuracies of classification with patch mean, pixel based maximum likelihood, ISODATA and ECHO were 0.735, 0.687, 0.610, and 0.605, respectively.;Spatial context was used to generate aggregate land cover information. An Urbanized Rate Index, defined based on the percentage of Building and Road area within a local window, was used to segment the image. Five summary landcover classes were identified from the Urbanized Rate segmentation and the image object classification: High Urbanized Rate and large building sizes, Intermediate Urbanized Rate and intermediate building sizes, Low urbanized rate and small building sizes, Forest, and Water

A Framework for Land Cover Classification Using Discrete Return LiDAR Data: Adopting Pseudo-Waveform and Hierarchical Segmentation

Acquiring current, accurate land-use information is critical for monitoring and understanding the impact of anthropogenic activities on natural environments.Remote sensing technologies are of increasing importance because of their capability to acquire information for large areas in a timely manner, enabling decision makers to be more effective in complex environments. Although optical imagery has demonstrated to be successful for land cover classification, active sensors, such as light detection and ranging (LiDAR), have distinct capabilities that can be exploited to improve classification results. However, utilization of LiDAR data for land cover classification has not been fully exploited. Moreover, spatial-spectral classification has recently gained significant attention since classification accuracy can be improved by extracting additional information from the neighboring pixels. Although spatial information has been widely used for spectral data, less attention has been given to LiDARdata. In this work, a new framework for land cover classification using discrete return LiDAR data is proposed. Pseudo-waveforms are generated from the LiDAR data and processed by hierarchical segmentation. Spatial featuresare extracted in a region-based way using a new unsupervised strategy for multiple pruning of the segmentation hierarchy. The proposed framework is validated experimentally on a real dataset acquired in an urban area. Better classification results are exhibited by the proposed framework compared to the cases in which basic LiDAR products such as digital surface model and intensity image are used. Moreover, the proposed region-based feature extraction strategy results in improved classification accuracies in comparison with a more traditional window-based approach

Information Extraction and Modeling from Remote Sensing Images: Application to the Enhancement of Digital Elevation Models

To deal with high complexity data such as remote sensing images presenting metric resolution over large areas, an innovative, fast and robust image processing system is presented. The modeling of increasing level of information is used to extract, represent and link image features to semantic content. The potential of the proposed techniques is demonstrated with an application to enhance and regularize digital elevation models based on information collected from RS images

REVIEW OF REMOTE SENSING IMAGE SEGMENTATION TECHNIQUES

Abstract Image segmentation is an important tool in image processing and can serve as an efficient front end to sophisticated algorithms and thereby simplify subsequent processing. Image segmentation by region merging follows a particular order in the choice of regions. The target of segmentation is always to simplify and/or change the representation of a graphic into something that is more meaningful and simpler to analyze. Image segmentation is normally used to locate objects and boundaries (lines, curves, etc.) in images. To improve segmentation accuracy and the correctness, this paper proposed a Dynamic Statistical Region Merging (DSRM) algorithm to find the automatically select scale value. The purpose of this paper is to gather various segmentation techniques that can be used for the segmentation of remote sensing images (RSI). The paper provides good starting for researchers to find automatically select scale value using the combination of DSRM and fuzzy logic

Illumination and Noise-Based Scene Classification - Application to SAR Sea Ice Imagery

Spatial intensity variation introduced by illumination changes is a challenging problem for image segmentation and classification. Many techniques have been proposed which focus on removing this illumination variation by estimating or modelling it. There is limited research on developing an illumination invariant classification technique which does not use any preprocessing. A major focus of this research is on automatically classifying synthetic aperture radar (SAR) images. These are large satellite images which pose many challenges for image classification including the incidence angle effect which is a strong illumination variation across the image. Mapping of full scene satellite images of sea-ice is important for navigational purposes for ships and also for climate research. The images obtained from the RADARSAT-2 satellite are dual band, high quality images. Currently, sea ice chart are produced manually by ice analysts at the Canadian Ice Service. However, this process can be automated to reduce processing time and obtain more detailed pixel-level ice maps. An automated classification algorithm to achieve sea ice and open water separation will greatly help the ice analyst by providing sufficient guidance in the initial stages of creating an ice map. It would also help the analyst to improve the accuracy while finding ice concentrations and remove subjective bias. The existing Iterative Region Growing by Semantics (IRGS) algorithm is not effective for full scene segmentation because of the incidence angle effect. This research proposes a "glocal" (global as well as local) approach to solve this problem. The image is divided in a rectangular grid and each rectangle is segmented using IRGS. This is viewed as an over-segmentation of the original image. Finally, IRGS is used globally to glue together the over-segmented regions. This method yields acceptable results with the denoised images. The proposed technique can also be used for general image classification purposes. Extensive testing was done to investigate the best set of parameters for the proposed approach. Images were simulated with the SAR illumination variation and multiplicative speckle noise. The technique was effective for general classification and attained accurate results for full scene SAR segmentation