Anisotropic Harmonic Analysis and Integration of Remotely Sensed Data

This thesis develops the theory of discrete directional Gabor frames and several algorithms for the analysis of remotely sensed image data, based on constructions of harmonic analysis. The problems of image registration, image superresolution, and image fusion are separate but interconnected; a general approach using transform methods is the focus of this thesis. The methods of geometric multiresolution analysis are explored, particularly those related to the shearlet transform. Using shearlets, a novel method of image registration is developed that aligns images based on their shearlet features. Additionally, the anisotropic nature of the shearlet transform is deployed to smoothly superrsolve remotely-sensed image with edge features. Wavelet packets, a generalization of wavelets, are utilized for a flexible image fusion algorithm. The interplay between theoretical guarantees for these mathematical constructions, and their effectiveness for image processing is explored throughout

Super Resolution of Wavelet-Encoded Images and Videos

In this dissertation, we address the multiframe super resolution reconstruction problem for wavelet-encoded images and videos. The goal of multiframe super resolution is to obtain one or more high resolution images by fusing a sequence of degraded or aliased low resolution images of the same scene. Since the low resolution images may be unaligned, a registration step is required before super resolution reconstruction. Therefore, we first explore in-band (i.e. in the wavelet-domain) image registration; then, investigate super resolution. Our motivation for analyzing the image registration and super resolution problems in the wavelet domain is the growing trend in wavelet-encoded imaging, and wavelet-encoding for image/video compression. Due to drawbacks of widely used discrete cosine transform in image and video compression, a considerable amount of literature is devoted to wavelet-based methods. However, since wavelets are shift-variant, existing methods cannot utilize wavelet subbands efficiently. In order to overcome this drawback, we establish and explore the direct relationship between the subbands under a translational shift, for image registration and super resolution. We then employ our devised in-band methodology, in a motion compensated video compression framework, to demonstrate the effective usage of wavelet subbands. Super resolution can also be used as a post-processing step in video compression in order to decrease the size of the video files to be compressed, with downsampling added as a pre-processing step. Therefore, we present a video compression scheme that utilizes super resolution to reconstruct the high frequency information lost during downsampling. In addition, super resolution is a crucial post-processing step for satellite imagery, due to the fact that it is hard to update imaging devices after a satellite is launched. Thus, we also demonstrate the usage of our devised methods in enhancing resolution of pansharpened multispectral images

Topics in Harmonic Analysis, Sparse Representations, and Data Analysis

Classical harmonic analysis has traditionally focused on linear and invertible transformations. Motivated by modern applications, there is a growing interest in non-linear analysis and synthesis operators. This thesis encompasses applications of computational harmonic analysis, with a strong emphasis on time-frequency methods, to modern problems arising in deep learning, data analysis, imaging, and signal processing. The first focus of this thesis deals with scattering transforms, which are particular realizations of convolutional neural networks. While the latter uses trained convolution kernels, scattering transforms use fixed ones, and this simplification allows mathematicians to develop a model of deep learning. Mallat originally introduced a wavelet scattering transform, but we study a complementary Fourier based version. We prove that the Fourier scattering transform enjoys properties that make it an effective feature extractor for classification, and we also construct a rotationally invariant modification of this transform. We provide experimental evidence that shows its effectiveness at representing complicated spectral data. The second focus of this thesis pertains to the mathematical foundations of super-resolution, which is concerned with the recovery of fine details from low-resolution observations. This imaging model can be mathematically formulated as an ill-posed inverse problem in the space of bounded complex measures. While the current theory primarily deals with the recovery of discrete measures with minimum separation greater than the Rayleigh length, we present alternative approaches. One direction exploits Beurling's results on minimal extrapolation to obtain a general theory that is pertinent to a wide class of measures, including those with geometric structure. Another approach is information theoretic and studies the min-max error for robust super-resolution of discrete measures below the Rayleigh length

Deep learning for inverse problems in remote sensing: super-resolution and SAR despeckling

L'abstract è presente nell'allegato / the abstract is in the attachmen

Feature Extraction and Design in Deep Learning Models

The selection and computation of meaningful features is critical for developing good deep learning methods. This dissertation demonstrates how focusing on this process can significantly improve the results of learning-based approaches. Specifically, this dissertation presents a series of different studies in which feature extraction and design was a significant factor for obtaining effective results. The first two studies are a content-based image retrieval system (CBIR) and a seagrass quantification study in which deep learning models were used to extract meaningful high-level features that significantly increased the performance of the approaches. Secondly, a method for change detection is proposed where the multispectral channels of satellite images are combined with different feature indices to improve the results. Then, two novel feature operators for mesh convolutional networks are presented that successfully extract invariant features from the faces and vertices of a mesh, respectively. The novel feature operators significantly outperform the previous state of the art for mesh classification and segmentation and provide two novel architectures for applying convolutional operations to the faces and vertices of geometric 3D meshes. Finally, a novel approach for automatic generation of 3D meshes is presented. The generative model efficiently uses the vertex-based feature operators proposed in the previous study and successfully learns to produce shapes from a mesh dataset with arbitrary topology

Feature extraction in image processing and deep learning

This thesis develops theoretical analysis of the approximation properties of neural networks, and algorithms to extract useful features of images in fields of deep learning, quantum energy regression and cancer image analysis. The separate applications are connected by using representation systems in harmonic analysis; we focus on deriving proper representations of data using Gabor transform in this thesis. A novel neural network with proven approximation properties dependent on its size is developed using Gabor system. In quantum energy regression, invariant representation of chemical molecules using electron densities is obtained based on the Gabor transform. Additionally, we dig into pooling functions, the feature extractor in deep neural networks, and develop a novel pooling strategy originated from the maximal function with stability property and stable performance. Anisotropic representation of data using the Shearlet transform is also explored in its ability to detect regions of interests of nuclei in cancer images