Image Restoration for Remote Sensing: Overview and Toolbox

Remote sensing provides valuable information about objects or areas from a distance in either active (e.g., RADAR and LiDAR) or passive (e.g., multispectral and hyperspectral) modes. The quality of data acquired by remotely sensed imaging sensors (both active and passive) is often degraded by a variety of noise types and artifacts. Image restoration, which is a vibrant field of research in the remote sensing community, is the task of recovering the true unknown image from the degraded observed image. Each imaging sensor induces unique noise types and artifacts into the observed image. This fact has led to the expansion of restoration techniques in different paths according to each sensor type. This review paper brings together the advances of image restoration techniques with particular focuses on synthetic aperture radar and hyperspectral images as the most active sub-fields of image restoration in the remote sensing community. We, therefore, provide a comprehensive, discipline-specific starting point for researchers at different levels (i.e., students, researchers, and senior researchers) willing to investigate the vibrant topic of data restoration by supplying sufficient detail and references. Additionally, this review paper accompanies a toolbox to provide a platform to encourage interested students and researchers in the field to further explore the restoration techniques and fast-forward the community. The toolboxes are provided in https://github.com/ImageRestorationToolbox.Comment: This paper is under review in GRS

Reinforcing Soft Independent Modelling of Class Analogy (SIMCA)

Soft independent modelling of class analogy (SIMCA) is a widely used subspacebased classification technique for spectral data analysis. The principal component (PC) subspace is built for each class separately through principal components analysis (PCA). The squared orthogonal distance (OD2) between the test sample and the class subspace of each class, and the squared score distance (SD2) between the projection of the test sample to the class subspace and the centre of the class subspace, are usually used in the classification rule of SIMCA to classify the test sample. Although it is commonly used to classify high-dimensional spectral data, SIMCA suffers from several drawbacks and some misleading calculations in literature. First, modelling classes separately makes the discriminative between-class information neglected. Second, the literature of SIMCA fail to explore the potential benefit of using geometric convex class models, whose superior classification performance has been demonstrated in face recognition. Third, based on our experiments on several real datasets, calculating OD2 using the formulae in a highlycited SIMCA paper (De Maesschalck et al., 1999) results in worse classification performance than using those in the original SIMCA paper (Wold, 1976) for some high-dimensional data and provides misleading classification results. Fourth, the distance metrics used in the classification rule of SIMCA are predetermined, which are not adapted to different data. Hence the research objectives of my PhD work are to reinforce SIMCA from the following four perspectives: O1) to make its feature space more discriminative; O2) to use geometric convex models as class models in SIMCA for spectral data classification and to study the classification mechanism of classification using different class models; O3) to investigate the equality and inequality of the calculations of OD2 in De Maesschalck et al. (1999) and Wold (1976) for low-dimensional and high-dimensional scenarios; and O4) to make its distance metric adaptively learned from data. In this thesis, we present four contributions to achieve the above four objectives, respectively: First, to achieve O1), we propose to first project the original data to a more discriminative subspace before applying SIMCA. To build such discriminative subspace, we propose the discriminatively ordered subspace (DOS) method, which selects the eigenvectors of the generating matrix with high discriminative ability between classes to span DOS. A paper of this work, “Building a discriminatively ordered subspace on the generating matrix to classify high-dimensional spectral data”, has been recently published by the journal of “Information Sciences”. Second, to achieve O2), we use the geometric convex models, convex hull and convex cone, as class models in SIMCA to classify spectral data. We study the dual of classification methods using three class models: the PC subspace, convex hull and convex cone, to investigate their classification mechanism. We provide theoretical results of the dual analysis, establish a separating hyperplane classification (SHC) framework and provide a new data exploration scheme to analyse the properties of a dataset and why such properties make one or more of the methods suitable for the data. Third, to achieve O3), we compare the calculations of OD2 in De Maesschalck et al. (1999) and Wold (1976). We show that the corresponding formulae in the two papers are equivalent, only when the training data of one class have more samples than features. When the training data of one class have more features than samples (i.e. high-dimensional), the formulae in De Maesschalck et al. (1999) are not precise and affect the classification results. Hence we suggest to use the formulae in Wold (1976) to calculate OD2, to get correct classification results of SIMCA for highdimensional data. Fourth, to achieve O4), we learn the distance metrics in SIMCA based on the derivation of a general formulation of the classification rules used in literature. We define the general formulation as the distance metric from a sample to a class subspace. We propose the method of learning distance to subspace to learn this distance metric by making the samples to be closer to their correct class subspaces while be farther away from their wrong class subspaces. Lastly, at the end of this thesis we append two pieces of work on hyperspectral image analysis. First, the joint paper with Mr Mingzhi Dong and Dr Jing-Hao Xue, “Spectral Nonlocal Restoration of Hyperspectral Images with Low-Rank Property”, has been published by the IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. Second, the joint paper with Dr Fei Zhou and Dr Jing-Hao Xue, “MvSSIM: A Quality Assessment Index for Hyperspectral Images”, has been in revision for Neurocomputing. As these two papers do not focus on the research objectives of this thesis, they are appended as some additional work during my PhD study

Adaptive Nonlocal Signal Restoration and Enhancement Techniques for High-Dimensional Data

The large number of practical applications involving digital images has motivated a significant interest towards restoration solutions that improve the visual quality of the data under the presence of various acquisition and compression artifacts. Digital images are the results of an acquisition process based on the measurement of a physical quantity of interest incident upon an imaging sensor over a specified period of time. The quantity of interest depends on the targeted imaging application. Common imaging sensors measure the number of photons impinging over a dense grid of photodetectors in order to produce an image similar to what is perceived by the human visual system. Different applications focus on the part of the electromagnetic spectrum not visible by the human visual system, and thus require different sensing technologies to form the image. In all cases, even with the advance of technology, raw data is invariably affected by a variety of inherent and external disturbing factors, such as the stochastic nature of the measurement processes or challenging sensing conditions, which may cause, e.g., noise, blur, geometrical distortion and color aberration. In this thesis we introduce two filtering frameworks for video and volumetric data restoration based on the BM3D grouping and collaborative filtering paradigm. In its general form, the BM3D paradigm leverages the correlation present within a nonlocal emph{group} composed of mutually similar basic filtering elements, e.g., patches, to attain an enhanced sparse representation of the group in a suitable transform domain where the energy of the meaningful part of the signal can be thus separated from that of the noise through coefficient shrinkage. We argue that the success of this approach largely depends on the form of the used basic filtering elements, which in turn define the subsequent spectral representation of the nonlocal group. Thus, the main contribution of this thesis consists in tailoring specific basic filtering elements to the the inherent characteristics of the processed data at hand. Specifically, we embed the local spatial correlation present in volumetric data through 3-D cubes, and the local spatial and temporal correlation present in videos through 3-D spatiotemporal volumes, i.e. sequences of 2-D blocks following a motion trajectory. The foundational aspect of this work is the analysis of the particular spectral representation of these elements. Specifically, our frameworks stack mutually similar 3-D patches along an additional fourth dimension, thus forming a 4-D data structure. By doing so, an effective group spectral description can be formed, as the phenomena acting along different dimensions in the data can be precisely localized along different spectral hyperplanes, and thus different filtering shrinkage strategies can be applied to different spectral coefficients to achieve the desired filtering results. This constitutes a decisive difference with the shrinkage traditionally employed in BM3D-algorithms, where different hyperplanes of the group spectrum are shrunk subject to the same degradation model. Different image processing problems rely on different observation models and typically require specific algorithms to filter the corrupted data. As a consequent contribution of this thesis, we show that our high-dimensional filtering model allows to target heterogeneous noise models, e.g., characterized by spatial and temporal correlation, signal-dependent distributions, spatially varying statistics, and non-white power spectral densities, without essential modifications to the algorithm structure. As a result, we develop state-of-the-art methods for a variety of fundamental image processing problems, such as denoising, deblocking, enhancement, deflickering, and reconstruction, which also find practical applications in consumer, medical, and thermal imaging