171 research outputs found
Image Deblurring and Super-resolution by Adaptive Sparse Domain Selection and Adaptive Regularization
As a powerful statistical image modeling technique, sparse representation has
been successfully used in various image restoration applications. The success
of sparse representation owes to the development of l1-norm optimization
techniques, and the fact that natural images are intrinsically sparse in some
domain. The image restoration quality largely depends on whether the employed
sparse domain can represent well the underlying image. Considering that the
contents can vary significantly across different images or different patches in
a single image, we propose to learn various sets of bases from a pre-collected
dataset of example image patches, and then for a given patch to be processed,
one set of bases are adaptively selected to characterize the local sparse
domain. We further introduce two adaptive regularization terms into the sparse
representation framework. First, a set of autoregressive (AR) models are
learned from the dataset of example image patches. The best fitted AR models to
a given patch are adaptively selected to regularize the image local structures.
Second, the image non-local self-similarity is introduced as another
regularization term. In addition, the sparsity regularization parameter is
adaptively estimated for better image restoration performance. Extensive
experiments on image deblurring and super-resolution validate that by using
adaptive sparse domain selection and adaptive regularization, the proposed
method achieves much better results than many state-of-the-art algorithms in
terms of both PSNR and visual perception.Comment: 35 pages. This paper is under review in IEEE TI
Astronomical Image Denoising Using Dictionary Learning
Astronomical images suffer a constant presence of multiple defects that are
consequences of the intrinsic properties of the acquisition equipments, and
atmospheric conditions. One of the most frequent defects in astronomical
imaging is the presence of additive noise which makes a denoising step
mandatory before processing data. During the last decade, a particular modeling
scheme, based on sparse representations, has drawn the attention of an ever
growing community of researchers. Sparse representations offer a promising
framework to many image and signal processing tasks, especially denoising and
restoration applications. At first, the harmonics, wavelets, and similar bases
and overcomplete representations have been considered as candidate domains to
seek the sparsest representation. A new generation of algorithms, based on
data-driven dictionaries, evolved rapidly and compete now with the
off-the-shelf fixed dictionaries. While designing a dictionary beforehand leans
on a guess of the most appropriate representative elementary forms and
functions, the dictionary learning framework offers to construct the dictionary
upon the data themselves, which provides us with a more flexible setup to
sparse modeling and allows to build more sophisticated dictionaries. In this
paper, we introduce the Centered Dictionary Learning (CDL) method and we study
its performances for astronomical image denoising. We show how CDL outperforms
wavelet or classic dictionary learning denoising techniques on astronomical
images, and we give a comparison of the effect of these different algorithms on
the photometry of the denoised images
SSIM-Inspired Quality Assessment, Compression, and Processing for Visual Communications
Objective Image and Video Quality Assessment (I/VQA) measures predict image/video quality as perceived by human beings - the ultimate consumers of visual data. Existing research in the area is mainly limited to benchmarking and monitoring of visual data. The use of I/VQA measures in the design and optimization of image/video processing algorithms and systems is more desirable, challenging and fruitful but has not been well explored. Among the recently proposed objective I/VQA approaches, the structural similarity (SSIM) index and its variants have emerged as promising measures that show superior performance as compared to the widely used mean squared error (MSE) and are computationally simple compared with other state-of-the-art perceptual quality measures. In addition, SSIM has a number of desirable mathematical properties for optimization tasks. The goal of this research is to break the tradition of using MSE as the optimization criterion for image and video processing algorithms. We tackle several important problems in visual communication applications by exploiting SSIM-inspired design and optimization to achieve significantly better performance.
Firstly, the original SSIM is a Full-Reference IQA (FR-IQA) measure that requires access to the original reference image, making it impractical in many visual communication applications. We propose a general purpose Reduced-Reference IQA (RR-IQA) method that can estimate SSIM with high accuracy with the help of a small number of RR features extracted from the original image. Furthermore, we introduce and demonstrate the novel idea of partially repairing an image using RR features. Secondly, image processing algorithms such as image de-noising and image super-resolution are required at various stages of visual communication systems, starting from image acquisition to image display at the receiver. We incorporate SSIM into the framework of sparse signal representation and non-local means methods and demonstrate improved performance in image de-noising and super-resolution. Thirdly, we incorporate SSIM into the framework of perceptual video compression. We propose an SSIM-based rate-distortion optimization scheme and an SSIM-inspired divisive optimization method that transforms the DCT domain frame residuals to a perceptually uniform space. Both approaches demonstrate the potential to largely improve the rate-distortion performance of state-of-the-art video codecs. Finally, in real-world visual communications, it is a common experience that end-users receive video with significantly time-varying quality due to the variations in video content/complexity, codec configuration, and network conditions. How human visual quality of experience (QoE) changes with such time-varying video quality is not yet well-understood. We propose a quality adaptation model that is asymmetrically tuned to increasing and decreasing quality. The model improves upon the direct SSIM approach in predicting subjective perceptual experience of time-varying video quality
Image Restoration for Remote Sensing: Overview and Toolbox
Remote sensing provides valuable information about objects or areas from a
distance in either active (e.g., RADAR and LiDAR) or passive (e.g.,
multispectral and hyperspectral) modes. The quality of data acquired by
remotely sensed imaging sensors (both active and passive) is often degraded by
a variety of noise types and artifacts. Image restoration, which is a vibrant
field of research in the remote sensing community, is the task of recovering
the true unknown image from the degraded observed image. Each imaging sensor
induces unique noise types and artifacts into the observed image. This fact has
led to the expansion of restoration techniques in different paths according to
each sensor type. This review paper brings together the advances of image
restoration techniques with particular focuses on synthetic aperture radar and
hyperspectral images as the most active sub-fields of image restoration in the
remote sensing community. We, therefore, provide a comprehensive,
discipline-specific starting point for researchers at different levels (i.e.,
students, researchers, and senior researchers) willing to investigate the
vibrant topic of data restoration by supplying sufficient detail and
references. Additionally, this review paper accompanies a toolbox to provide a
platform to encourage interested students and researchers in the field to
further explore the restoration techniques and fast-forward the community. The
toolboxes are provided in https://github.com/ImageRestorationToolbox.Comment: This paper is under review in GRS
Real-time Ultrasound Signals Processing: Denoising and Super-resolution
Ultrasound acquisition is widespread in the biomedical field, due to its properties of low cost, portability, and non-invasiveness for the patient. The processing and analysis of US signals, such as images, 2D videos, and volumetric images, allows the physician to monitor the evolution of the patient's disease, and support diagnosis, and treatments (e.g., surgery). US images are affected by speckle noise, generated by the overlap of US waves. Furthermore, low-resolution images are acquired when a high acquisition frequency is applied to accurately characterise the behaviour of anatomical features that quickly change over time. Denoising and super-resolution of US signals are relevant to improve the visual evaluation of the physician and the performance and accuracy of processing methods, such as segmentation and classification. The main requirements for the processing and analysis of US signals are real-time execution, preservation of anatomical features, and reduction of artefacts. In this context, we present a novel framework for the real-time denoising of US 2D images based on deep learning and high-performance computing, which reduces noise while preserving anatomical features in real-time execution. We extend our framework to the denoise of arbitrary US signals, such as 2D videos and 3D images, and we apply denoising algorithms that account for spatio-temporal signal properties into an image-to-image deep learning model. As a building block of this framework, we propose a novel denoising method belonging to the class of low-rank approximations, which learns and predicts the optimal thresholds of the Singular Value Decomposition. While previous denoise work compromises the computational cost and effectiveness of the method, the proposed framework achieves the results of the best denoising algorithms in terms of noise removal, anatomical feature preservation, and geometric and texture properties conservation, in a real-time execution that respects industrial constraints. The framework reduces the artefacts (e.g., blurring) and preserves the spatio-temporal consistency among frames/slices; also, it is general to the denoising algorithm, anatomical district, and noise intensity. Then, we introduce a novel framework for the real-time reconstruction of the non-acquired scan lines through an interpolating method; a deep learning model improves the results of the interpolation to match the target image (i.e., the high-resolution image). We improve the accuracy of the prediction of the reconstructed lines through the design of the network architecture and the loss function. %The design of the deep learning architecture and the loss function allow the network to improve the accuracy of the prediction of the reconstructed lines. In the context of signal approximation, we introduce our kernel-based sampling method for the reconstruction of 2D and 3D signals defined on regular and irregular grids, with an application to US 2D and 3D images. Our method improves previous work in terms of sampling quality, approximation accuracy, and geometry reconstruction with a slightly higher computational cost. For both denoising and super-resolution, we evaluate the compliance with the real-time requirement of US applications in the medical domain and provide a quantitative evaluation of denoising and super-resolution methods on US and synthetic images. Finally, we discuss the role of denoising and super-resolution as pre-processing steps for segmentation and predictive analysis of breast pathologies
Adaptive Representations for Image Restoration
In the �eld of image processing, building good representation models for
natural images is crucial for various applications, such as image restora-
tion, sampling, segmentation, etc. Adaptive image representation models
are designed for describing the intrinsic structures of natural images. In
the classical Bayesian inference, this representation is often known as the
prior of the intensity distribution of the input image. Early image priors
have forms such as total variation norm, Markov Random Fields (MRF),
and wavelets. Recently, image priors obtained from machine learning tech-
niques tend to be more adaptive, which aims at capturing the natural image
models via learning from larger databases. In this thesis, we study adaptive
representations of natural images for image restoration.
The purpose of image restoration is to remove the artifacts which degrade
an image. The degradation comes in many forms such as image blurs,
noises, and artifacts from the codec. Take image denoising for an example.
There are several classic representation methods which can generate state-
of-the-art results. The �rst one is the assumption of image self-similarity.
However, this representation has the issue that sometimes the self-similarity
assumption would fail because of high noise levels or unique image contents.
The second one is the wavelet based nonlocal representation, which also has
a problem in that the �xed basis function is not adaptive enough for any
arbitrary type of input images. The third is the sparse coding using over-
complete dictionaries, which does not have the hierarchical structure that is
similar to the one in human visual system and is therefore prone to denoising
artifacts.
My research started from image denoising. Through the thorough review
and evaluation of state-of-the-art denoising methods, it was found that the representation of images is substantially important for the denoising tech-
nique. At the same time, an improvement on one of the nonlocal denoising
method was proposed, which improves the representation of images by the
integration of Gaussian blur, clustering and Rotationally Invariant Block
Matching. Enlightened by the successful application of sparse coding in
compressive sensing, we exploited the image self-similarity by using a sparse
representation based on wavelet coe�cients in a nonlocal and hierarchical
way, which generates competitive results compared to the state-of-the-art
denoising algorithms. Meanwhile, another adaptive local �lter learned by
Genetic Programming (GP) was proposed for e�cient image denoising. In
this work, we employed GP to �nd the optimal representations for local im-
age patches through training on massive datasets, which yields competitive
results compared to state-of-the-art local denoising �lters. After success-
fully dealt with the denoising part, we moved to the parameter estimation
for image degradation models. For instance, image blur identi�cation uses
deep learning, which has recently been proposed as a popular image repre-
sentation approach. This work has also been extended to blur estimation
based on the fact that the second step of the framework has been replaced
with general regression neural network. In a word, in this thesis, spatial cor-
relations, sparse coding, genetic programming, deep learning are explored
as adaptive image representation models for both image restoration and
parameter estimation.
We conclude this thesis by considering methods based on machine learning
to be the best adaptive representations for natural images. We have shown
that they can generate better results than conventional representation mod-
els for the tasks of image denoising and deblurring
- …