Image Restoration for Remote Sensing: Overview and Toolbox

Remote sensing provides valuable information about objects or areas from a distance in either active (e.g., RADAR and LiDAR) or passive (e.g., multispectral and hyperspectral) modes. The quality of data acquired by remotely sensed imaging sensors (both active and passive) is often degraded by a variety of noise types and artifacts. Image restoration, which is a vibrant field of research in the remote sensing community, is the task of recovering the true unknown image from the degraded observed image. Each imaging sensor induces unique noise types and artifacts into the observed image. This fact has led to the expansion of restoration techniques in different paths according to each sensor type. This review paper brings together the advances of image restoration techniques with particular focuses on synthetic aperture radar and hyperspectral images as the most active sub-fields of image restoration in the remote sensing community. We, therefore, provide a comprehensive, discipline-specific starting point for researchers at different levels (i.e., students, researchers, and senior researchers) willing to investigate the vibrant topic of data restoration by supplying sufficient detail and references. Additionally, this review paper accompanies a toolbox to provide a platform to encourage interested students and researchers in the field to further explore the restoration techniques and fast-forward the community. The toolboxes are provided in https://github.com/ImageRestorationToolbox.Comment: This paper is under review in GRS

Nonlinear Adaptive Diffusion Models for Image Denoising

Most of digital image applications demand on high image quality. Unfortunately, images often are degraded by noise during the formation, transmission, and recording processes. Hence, image denoising is an essential processing step preceding visual and automated analyses. Image denoising methods can reduce image contrast, create block or ring artifacts in the process of denoising. In this dissertation, we develop high performance non-linear diffusion based image denoising methods, capable to preserve edges and maintain high visual quality. This is attained by different approaches: First, a nonlinear diffusion is presented with robust M-estimators as diffusivity functions. Secondly, the knowledge of textons derived from Local Binary Patterns (LBP) which unify divergent statistical and structural models of the region analysis is utilized to adjust the time step of diffusion process. Next, the role of nonlinear diffusion which is adaptive to the local context in the wavelet domain is investigated, and the stationary wavelet context based diffusion (SWCD) is developed for performing the iterative shrinkage. Finally, we develop a locally- and feature-adaptive diffusion (LFAD) method, where each image patch/region is diffused individually, and the diffusivity function is modified to incorporate the Inverse Difference Moment as a local estimate of the gradient. Experiments have been conducted to evaluate the performance of each of the developed method and compare it to the reference group and to the state-of-the-art methods

BM3D Image Denoising using Learning-Based Adaptive Hard Thresholding

Image denoising is an important pre-processing step in most imaging applications. Block Matching and 3D Filtering (BM3D) is considered to be the current state-of-art algorithm for additive image denoising. But this algorithm uses a fixed hard thresholding scheme to attenuate noise from a 3D block. Experiments show that this fixed hard thresholding deteriorates the performance of BM3D because it does not consider the context of corresponding blocks. In this thesis, we propose a learning based adaptive hard thresholding method to solve this issue. Also, BM3D algorithm requires as an input the value of the noise level in the input image. But in real life it is not practical to pass as an input such noise level. In this thesis, we also attempt to automatically estimate the level of the noise in the input image. Experimental results demonstrate that our proposed algorithm outperforms BM3D in both objective and subjective fidelity criteria

Statistical Methods for Image Registration and Denoising

This dissertation describes research into image processing techniques that enhance military operational and support activities. The research extends existing work on image registration by introducing a novel method that exploits local correlations to improve the performance of projection-based image registration algorithms. The dissertation also extends the bounds on image registration performance for both projection-based and full-frame image registration algorithms and extends the Barankin bound from the one-dimensional case to the problem of two-dimensional image registration. It is demonstrated that in some instances, the Cramer-Rao lower bound is an overly-optimistic predictor of image registration performance and that under some conditions, the Barankin bound is a better predictor of shift estimator performance. The research also looks at the related problem of single-frame image denoising using block-based methods. The research introduces three algorithms that operate by identifying regions of interest within a noise-corrupted image and then generating noise free estimates of the regions as averages of similar regions in the image

Adaptive Representations for Image Restoration

In the �eld of image processing, building good representation models for natural images is crucial for various applications, such as image restora- tion, sampling, segmentation, etc. Adaptive image representation models are designed for describing the intrinsic structures of natural images. In the classical Bayesian inference, this representation is often known as the prior of the intensity distribution of the input image. Early image priors have forms such as total variation norm, Markov Random Fields (MRF), and wavelets. Recently, image priors obtained from machine learning tech- niques tend to be more adaptive, which aims at capturing the natural image models via learning from larger databases. In this thesis, we study adaptive representations of natural images for image restoration. The purpose of image restoration is to remove the artifacts which degrade an image. The degradation comes in many forms such as image blurs, noises, and artifacts from the codec. Take image denoising for an example. There are several classic representation methods which can generate state- of-the-art results. The �rst one is the assumption of image self-similarity. However, this representation has the issue that sometimes the self-similarity assumption would fail because of high noise levels or unique image contents. The second one is the wavelet based nonlocal representation, which also has a problem in that the �xed basis function is not adaptive enough for any arbitrary type of input images. The third is the sparse coding using over- complete dictionaries, which does not have the hierarchical structure that is similar to the one in human visual system and is therefore prone to denoising artifacts. My research started from image denoising. Through the thorough review and evaluation of state-of-the-art denoising methods, it was found that the representation of images is substantially important for the denoising tech- nique. At the same time, an improvement on one of the nonlocal denoising method was proposed, which improves the representation of images by the integration of Gaussian blur, clustering and Rotationally Invariant Block Matching. Enlightened by the successful application of sparse coding in compressive sensing, we exploited the image self-similarity by using a sparse representation based on wavelet coe�cients in a nonlocal and hierarchical way, which generates competitive results compared to the state-of-the-art denoising algorithms. Meanwhile, another adaptive local �lter learned by Genetic Programming (GP) was proposed for e�cient image denoising. In this work, we employed GP to �nd the optimal representations for local im- age patches through training on massive datasets, which yields competitive results compared to state-of-the-art local denoising �lters. After success- fully dealt with the denoising part, we moved to the parameter estimation for image degradation models. For instance, image blur identi�cation uses deep learning, which has recently been proposed as a popular image repre- sentation approach. This work has also been extended to blur estimation based on the fact that the second step of the framework has been replaced with general regression neural network. In a word, in this thesis, spatial cor- relations, sparse coding, genetic programming, deep learning are explored as adaptive image representation models for both image restoration and parameter estimation. We conclude this thesis by considering methods based on machine learning to be the best adaptive representations for natural images. We have shown that they can generate better results than conventional representation mod- els for the tasks of image denoising and deblurring

Robust density modelling using the student's t-distribution for human action recognition

The extraction of human features from videos is often inaccurate and prone to outliers. Such outliers can severely affect density modelling when the Gaussian distribution is used as the model since it is highly sensitive to outliers. The Gaussian distribution is also often used as base component of graphical models for recognising human actions in the videos (hidden Markov model and others) and the presence of outliers can significantly affect the recognition accuracy. In contrast, the Student's t-distribution is more robust to outliers and can be exploited to improve the recognition rate in the presence of abnormal data. In this paper, we present an HMM which uses mixtures of t-distributions as observation probabilities and show how experiments over two well-known datasets (Weizmann, MuHAVi) reported a remarkable improvement in classification accuracy. © 2011 IEEE

Anatomy Segmentation of Breast Ultrasound images

Breast cancer is one of the most common cancers in women, affecting hundreds of women. Even though the detection of cancer has been largely studied, the decision of which strategy to take concerning oncoplastic surgery still relies almost exclusively on the surgeon's perception of post-surgical aesthetic result, which sometime leads to unsatisfactory outcomes. In order to empower the patients on the joint decision process there needs to exist a better communication between the parts. This can be achieved by developing medical grade 3D models of the breast and explaining better the surgical options and their results. In order to obtain such models, some effort has been made concerning multi-modality radiological imaging combination. This line of research has yet to mature. In turn, the modality alignment requires accurate landmarks to be produced. 2D Ultrasound imaging has not been sufficiently studied for multimodal registration due to the image characteristics and thus, landmark segmentation is of utmost importance. This task can be challenging since US data presents high specular noise levels and the presence of some tissues alters the perception of other tissues. Objectives: ● Study and evaluation of different techniques for anatomical landmark segmentation, such as Skin, Fat and Glandular tissue, Lesions (masses and cysts), Pectoral muscle; ● Development of Ultrasound segmentation methods for acquiring landmarks; ● Evaluation of the developed methods with manual annotations and comparison of results with the current algorithm alternatives.Breast cancer is one of the most common cancers in women, affecting hundreds of women. Even though the detection of cancer has been largely studied, the decision of which strategy to take concerning oncoplastic surgery still relies almost exclusively on the surgeon's perception of post-surgical aesthetic result, which sometime leads to unsatisfactory outcomes. In order to empower the patients on the joint decision process there needs to exist a better communication between the parts. This can be achieved by developing medical grade 3D models of the breast and explaining better the surgical options and their results. In order to obtain such models, some effort has been made concerning multi-modality radiological imaging combination. This line of research has yet to mature. In turn, the modality alignment requires accurate landmarks to be produced. 2D Ultrasound imaging has not been sufficiently studied for multimodal registration due to the image characteristics and thus, landmark segmentation is of utmost importance. This task can be challenging since US data presents high specular noise levels and the presence of some tissues alters the perception of other tissues. Objectives: ● Study and evaluation of different techniques for anatomical landmark segmentation, such as Skin, Fat and Glandular tissue, Lesions (masses and cysts), Pectoral muscle; ● Development of Ultrasound segmentation methods for acquiring landmarks; ● Evaluation of the developed methods with manual annotations and comparison of results with the current algorithm alternatives

Robust computational intelligence techniques for visual information processing

The third part is exclusively dedicated to the super-resolution of Magnetic Resonance Images. In one of these works, an algorithm based on the random shifting technique is developed. Besides, we studied noise removal and resolution enhancement simultaneously. To end, the cost function of deep networks has been modified by different combinations of norms in order to improve their training. Finally, the general conclusions of the research are presented and discussed, as well as the possible future research lines that are able to make use of the results obtained in this Ph.D. thesis.This Ph.D. thesis is about image processing by computational intelligence techniques. Firstly, a general overview of this book is carried out, where the motivation, the hypothesis, the objectives, and the methodology employed are described. The use and analysis of different mathematical norms will be our goal. After that, state of the art focused on the applications of the image processing proposals is presented. In addition, the fundamentals of the image modalities, with particular attention to magnetic resonance, and the learning techniques used in this research, mainly based on neural networks, are summarized. To end up, the mathematical framework on which this work is based on, ₚ-norms, is defined. Three different parts associated with image processing techniques follow. The first non-introductory part of this book collects the developments which are about image segmentation. Two of them are applications for video surveillance tasks and try to model the background of a scenario using a specific camera. The other work is centered on the medical field, where the goal of segmenting diabetic wounds of a very heterogeneous dataset is addressed. The second part is focused on the optimization and implementation of new models for curve and surface fitting in two and three dimensions, respectively. The first work presents a parabola fitting algorithm based on the measurement of the distances of the interior and exterior points to the focus and the directrix. The second work changes to an ellipse shape, and it ensembles the information of multiple fitting methods. Last, the ellipsoid problem is addressed in a similar way to the parabola