Active Learning: Any Value for Classification of Remotely Sensed Data?

Active learning, which has a strong impact on processing data prior to the classification phase, is an active research area within the machine learning community, and is now being extended for remote sensing applications. To be effective, classification must rely on the most informative pixels, while the training set should be as compact as possible. Active learning heuristics provide capability to select unlabeled data that are the “most informative” and to obtain the respective labels, contributing to both goals. Characteristics of remotely sensed image data provide both challenges and opportunities to exploit the potential advantages of active learning. We present an overview of active learning methods, then review the latest techniques proposed to cope with the problem of interactive sampling of training pixels for classification of remotely sensed data with support vector machines (SVMs). We discuss remote sensing specific approaches dealing with multisource and spatially and time-varying data, and provide examples for high-dimensional hyperspectral imagery

Multisource and Multitemporal Data Fusion in Remote Sensing

The sharp and recent increase in the availability of data captured by different sensors combined with their considerably heterogeneous natures poses a serious challenge for the effective and efficient processing of remotely sensed data. Such an increase in remote sensing and ancillary datasets, however, opens up the possibility of utilizing multimodal datasets in a joint manner to further improve the performance of the processing approaches with respect to the application at hand. Multisource data fusion has, therefore, received enormous attention from researchers worldwide for a wide variety of applications. Moreover, thanks to the revisit capability of several spaceborne sensors, the integration of the temporal information with the spatial and/or spectral/backscattering information of the remotely sensed data is possible and helps to move from a representation of 2D/3D data to 4D data structures, where the time variable adds new information as well as challenges for the information extraction algorithms. There are a huge number of research works dedicated to multisource and multitemporal data fusion, but the methods for the fusion of different modalities have expanded in different paths according to each research community. This paper brings together the advances of multisource and multitemporal data fusion approaches with respect to different research communities and provides a thorough and discipline-specific starting point for researchers at different levels (i.e., students, researchers, and senior researchers) willing to conduct novel investigations on this challenging topic by supplying sufficient detail and references

Domain Adaptation in remote sensing: increasing the portability of land-cover classifiers

Among the types of remote sensing acquisitions, optical images are certainly one of the most widely relied upon data sources for Earth observation. They provide detailed measurements of the electromagnetic radiation reflected or emitted by each pixel in the scene. Through a process termed supervised land-cover classification, this allows to automatically yet accurately distinguish objects at the surface of our planet. In this respect, when producing a land-cover map of the surveyed area, the availability of training examples representative of each thematic class is crucial for the success of the classification procedure. However, in real applications, due to several constraints on the sample collection process, labeled pixels are usually scarce. When analyzing an image for which those key samples are unavailable, a viable solution consists in resorting to the ground truth data of other previously acquired images. This option is attractive but several factors such as atmospheric, ground and acquisition conditions can cause radiometric differences between the images, hindering therefore the transfer of knowledge from one image to another. The goal of this Thesis is to supply remote sensing image analysts with suitable processing techniques to ensure a robust portability of the classification models across different images. The ultimate purpose is to map the land-cover classes over large spatial and temporal extents with minimal ground information. To overcome, or simply quantify, the observed shifts in the statistical distribution of the spectra of the materials, we study four approaches issued from the field of machine learning. First, we propose a strategy to intelligently sample the image of interest to collect the labels only in correspondence of the most useful pixels. This iterative routine is based on a constant evaluation of the pertinence to the new image of the initial training data actually belonging to a different image. Second, an approach to reduce the radiometric differences among the images by projecting the respective pixels in a common new data space is presented. We analyze a kernel-based feature extraction framework suited for such problems, showing that, after this relative normalization, the cross-image generalization abilities of a classifier are highly increased. Third, we test a new data-driven measure of distance between probability distributions to assess the distortions caused by differences in the acquisition geometry affecting series of multi-angle images. Also, we gauge the portability of classification models through the sequences. In both exercises, the efficacy of classic physically- and statistically-based normalization methods is discussed. Finally, we explore a new family of approaches based on sparse representations of the samples to reciprocally convert the data space of two images. The projection function bridging the images allows a synthesis of new pixels with more similar characteristics ultimately facilitating the land-cover mapping across images

Evaluation of Machine Learning Algorithms for Lake Ice Classification from Optical Remote Sensing Data

The topic of lake ice cover mapping from satellite remote sensing data has gained interest in recent years since it allows the extent of lake ice and the dynamics of ice phenology over large areas to be monitored. Mapping lake ice extent can record the loss of the perennial ice cover for lakes located in the High Arctic. Moreover, ice phenology dates, retrieved from lake ice maps, are useful for assessing long-term trends and variability in climate, particularly due to their sensitivity to changes in near-surface air temperature. However, existing knowledge-driven (threshold-based) retrieval algorithms for lake ice-water classification that use top-of-the-atmosphere (TOA) reflectance products do not perform well under the condition of large solar zenith angles, resulting in low TOA reflectance. Machine learning (ML) techniques have received considerable attention in the remote sensing field for the past several decades, but they have not yet been applied in lake ice classification from optical remote sensing imagery. Therefore, this research has evaluated the capability of ML classifiers to enhance lake ice mapping using multispectral optical remote sensing data (MODIS L1B (TOA) product). Chapter 3, the main manuscript of this thesis, presents an investigation of four ML classifiers (i.e. multinomial logistic regression, MLR; support vector machine, SVM; random forest, RF; gradient boosting trees, GBT) in lake ice classification. Results are reported using 17 lakes located in the Northern Hemisphere, which represent different characteristics regarding area, altitude, freezing frequency, and ice cover duration. According to the overall accuracy assessment using a random k-fold cross-validation (k = 100), all ML classifiers were able to produce classification accuracies above 94%, and RF and GBT provided above 98% classification accuracies. Moreover, the RF and GBT algorithms provided a more visually accurate depiction of lake ice cover under challenging conditions (i.e., high solar zenith angles, black ice, and thin cloud cover). The two tree-based classifiers were found to provide the most robust spatial transferability over the 17 lakes and performed consistently well across three ice seasons, better than the other classifiers. Moreover, RF was insensitive to the choice of the hyperparameters compared to the other three classifiers. The results demonstrate that RF and GBT provide a great potential to map accurately lake ice cover globally over a long time-series. Additionally, a case study applying a convolution neural network (CNN) model for ice classification in Great Slave Lake, Canada is presented in Appendix A. Eighteen images acquired during the the ice season of 2009-2010 were used in this study. The proposed CNN produced a 98.03% accuracy with the testing dataset; however, the accuracy dropped to 90.13% using an independent (out-of-sample) validation dataset. Results show the powerful learning performance of the proposed CNN with the testing data accuracy obtained. At the same time, the accuracy reduction of the validation dataset indicates the overfitting behavior of the proposed model. A follow-up investigation would be needed to improve its performance. This thesis investigated the capability of ML algorithms (both pixel-based and spatial-based) in lake ice classification from the MODIS L1B product. Overall, ML techniques showed promising performances for lake ice cover mapping from the optical remote sensing data. The tree-based classifiers (pixel-based) exhibited the potential to produce accurate lake ice classification at a large-scale over long time-series. In addition, more work would be of benefit for improving the application of CNN in lake ice cover mapping from optical remote sensing imagery

Target classification in multimodal video

The presented thesis focuses on enhancing scene segmentation and target recognition methodologies via the mobilisation of contextual information. The algorithms developed to achieve this goal utilise multi-modal sensor information collected across varying scenarios, from controlled indoor sequences to challenging rural locations. Sensors are chiefly colour band and long wave infrared (LWIR), enabling persistent surveillance capabilities across all environments. In the drive to develop effectual algorithms towards the outlined goals, key obstacles are identified and examined: the recovery of background scene structure from foreground object ’clutter’, employing contextual foreground knowledge to circumvent training a classifier when labeled data is not readily available, creating a labeled LWIR dataset to train a convolutional neural network (CNN) based object classifier and the viability of spatial context to address long range target classification when big data solutions are not enough. For an environment displaying frequent foreground clutter, such as a busy train station, we propose an algorithm exploiting foreground object presence to segment underlying scene structure that is not often visible. If such a location is outdoors and surveyed by an infra-red (IR) and visible band camera set-up, scene context and contextual knowledge transfer allows reasonable class predictions for thermal signatures within the scene to be determined. Furthermore, a labeled LWIR image corpus is created to train an infrared object classifier, using a CNN approach. The trained network demonstrates effective classification accuracy of 95% over 6 object classes. However, performance is not sustainable for IR targets acquired at long range due to low signal quality and classification accuracy drops. This is addressed by mobilising spatial context to affect network class scores, restoring robust classification capability

Data fusion by using machine learning and computational intelligence techniques for medical image analysis and classification

Data fusion is the process of integrating information from multiple sources to produce specific, comprehensive, unified data about an entity. Data fusion is categorized as low level, feature level and decision level. This research is focused on both investigating and developing feature- and decision-level data fusion for automated image analysis and classification. The common procedure for solving these problems can be described as: 1) process image for region of interest\u27 detection, 2) extract features from the region of interest and 3) create learning model based on the feature data. Image processing techniques were performed using edge detection, a histogram threshold and a color drop algorithm to determine the region of interest. The extracted features were low-level features, including textual, color and symmetrical features. For image analysis and classification, feature- and decision-level data fusion techniques are investigated for model learning using and integrating computational intelligence and machine learning techniques. These techniques include artificial neural networks, evolutionary algorithms, particle swarm optimization, decision tree, clustering algorithms, fuzzy logic inference, and voting algorithms. This work presents both the investigation and development of data fusion techniques for the application areas of dermoscopy skin lesion discrimination, content-based image retrieval, and graphic image type classification --Abstract, page v