Graph Spectral Image Processing

Recent advent of graph signal processing (GSP) has spurred intensive studies of signals that live naturally on irregular data kernels described by graphs (e.g., social networks, wireless sensor networks). Though a digital image contains pixels that reside on a regularly sampled 2D grid, if one can design an appropriate underlying graph connecting pixels with weights that reflect the image structure, then one can interpret the image (or image patch) as a signal on a graph, and apply GSP tools for processing and analysis of the signal in graph spectral domain. In this article, we overview recent graph spectral techniques in GSP specifically for image / video processing. The topics covered include image compression, image restoration, image filtering and image segmentation

A Tutorial on Speckle Reduction in Synthetic Aperture Radar Images

Speckle is a granular disturbance, usually modeled as a multiplicative noise, that affects synthetic aperture radar (SAR) images, as well as all coherent images. Over the last three decades, several methods have been proposed for the reduction of speckle, or despeckling, in SAR images. Goal of this paper is making a comprehensive review of despeckling methods since their birth, over thirty years ago, highlighting trends and changing approaches over years. The concept of fully developed speckle is explained. Drawbacks of homomorphic filtering are pointed out. Assets of multiresolution despeckling, as opposite to spatial-domain despeckling, are highlighted. Also advantages of undecimated, or stationary, wavelet transforms over decimated ones are discussed. Bayesian estimators and probability density function (pdf) models in both spatial and multiresolution domains are reviewed. Scale-space varying pdf models, as opposite to scale varying models, are promoted. Promising methods following non-Bayesian approaches, like nonlocal (NL) filtering and total variation (TV) regularization, are reviewed and compared to spatial- and wavelet-domain Bayesian filters. Both established and new trends for assessment of despeckling are presented. A few experiments on simulated data and real COSMO-SkyMed SAR images highlight, on one side the costperformance tradeoff of the different methods, on the other side the effectiveness of solutions purposely designed for SAR heterogeneity and not fully developed speckle. Eventually, upcoming methods based on new concepts of signal processing, like compressive sensing, are foreseen as a new generation of despeckling, after spatial-domain and multiresolution-domain method

Wavelet/shearlet hybridized neural networks for biomedical image restoration

Recently, new programming paradigms have emerged that combine parallelism and numerical computations with algorithmic differentiation. This approach allows for the hybridization of neural network techniques for inverse imaging problems with more traditional methods such as wavelet-based sparsity modelling techniques. The benefits are twofold: on the one hand traditional methods with well-known properties can be integrated in neural networks, either as separate layers or tightly integrated in the network, on the other hand, parameters in traditional methods can be trained end-to-end from datasets in a neural network "fashion" (e.g., using Adagrad or Adam optimizers). In this paper, we explore these hybrid neural networks in the context of shearlet-based regularization for the purpose of biomedical image restoration. Due to the reduced number of parameters, this approach seems a promising strategy especially when dealing with small training data sets

Scale-aware decomposition of images based on patch-based filtering

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2015. 2. 조남익.This dissertation presents an image decomposition algorithm based on patch-based filtering, for splitting an image into a structure layer and a texture layer. There are many applications through the decomposition because each layer can be processed respectively and appropriate manipulations are accomplished. Generally, structure layer captures coarse structure with large discontinuities and a texture layer contains fine details or proper patterns. The image decomposition is done by edge-preserving smoothing where structure layer can be obtained by applying smoothing filters to an image and then a texture layer by subtracting the filtered image from the original. The main contribution of this dissertation is to design an efficient and effective edge-preserving filter that can be adapted to various scales of images. The advantage of the proposed decomposition scheme is that it is robust to noise and can be extended to a noisy image decomposition, while conventional image decomposition methods cannot be applied to a noisy image decomposition and conventional image denoising methods are not suitable for image decomposition. To be specific, a patch-based framework is proposed in this dissertation, which is efficient in image denoising and it is designed to smooth an image while preserving details and texture. Specifically, given a pixel, the filtering output is computed as the weighted average of neighboring pixels. For computing the weights, a set of similar patches is found at each pixel by considering patch similarities based on mean squared error (MSE) and other constraints. Then, weights between each patch and its similar patches are computed respectively. With the patch weights, all the pixels in a patch are updated at the same time while adapting to the local pixel weight. For better edge-preserving smoothing, the proposed algorithm utilizes two iterations which are performed through the same smoothing filter with different parameters. Also kernel bandwidth and the number of similar patches are tuned for multi-scale image decomposition. The proposed decomposition can be applied to many applications, such as HDR tone mapping, detail enhancement, image denoising, and image coding, etc. In detail enhancement, the proposed smoothing filter is utilized to extract image detail and enhance it. In HDR tone mapping, a typical framework is used where the smoothing operator is replaced by the proposed one to reduce contrast range of a high dynamic range image to display it on low dynamic range devices. For image denoising, a noisy input is decomposed into structure/texture/noise and the noise layer is discarded while the texture layer is restored through the histogram matching. Also a novel coding scheme named as ``structure scalable image coding scheme'' is proposed where structure layer and salient texture layer are encoded for efficient image coding. Experimental results show that the proposed framework works well for image decomposition and it is robust to the presence of noise. Also it is verified that the proposed work can be utilized in many applications. In addition, by adopting the proposed method in decomposition of a noisy image, both image denoising and image enhancement can be achieved in the proposed framework. Furthermore, the proposed image coding method reduces compression artifact and improve the performance of image coding.Abstract i Contents iv List of Figures vi List of Tables xi 1 Introduction 1 1.1 Image decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Image enhancement . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Image denoising . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3.1 Spatial denoising . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3.2 Transformdomain denoising . . . . . . . . . . . . . . . . . . 9 1.3.3 benefits of combined image decomposition and image denoising 9 1.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2 Related work 17 2.1 Image decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.1.1 Laplacian subbands . . . . . . . . . . . . . . . . . . . . . . . 17 2.2 Edge-preserving smoothing . . . . . . . . . . . . . . . . . . . . . . . 18 2.2.1 Bilateral filtering . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.2.2 Nonlocal means filtering . . . . . . . . . . . . . . . . . . . . . 21 3 Scale-aware decomposition of images based on patch-based filtering 23 3.1 Edge-preserving smoothing via patch-based framework . . . . . . . . 23 3.2 Multi-scale image decomposition . . . . . . . . . . . . . . . . . . . . 26 4 Applications 31 4.1 Image enhancement . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.1.1 Detail enhancement . . . . . . . . . . . . . . . . . . . . . . . 31 4.1.2 HDR tone mapping . . . . . . . . . . . . . . . . . . . . . . . 36 4.2 Image denoising . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.2.1 A noisy image decomposition . . . . . . . . . . . . . . . . . . 40 4.2.2 texture enhancement via histogram preservation . . . . . . . 41 4.2.3 image denoising via subband BLF . . . . . . . . . . . . . . . 44 4.2.4 Experimental results of image denoising . . . . . . . . . . . . 48 4.3 Image coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.3.1 Structure scalable image coding framework . . . . . . . . . . 61 5 Conclusion 73 Bibliography 76Docto

Convolutional Deblurring for Natural Imaging

In this paper, we propose a novel design of image deblurring in the form of one-shot convolution filtering that can directly convolve with naturally blurred images for restoration. The problem of optical blurring is a common disadvantage to many imaging applications that suffer from optical imperfections. Despite numerous deconvolution methods that blindly estimate blurring in either inclusive or exclusive forms, they are practically challenging due to high computational cost and low image reconstruction quality. Both conditions of high accuracy and high speed are prerequisites for high-throughput imaging platforms in digital archiving. In such platforms, deblurring is required after image acquisition before being stored, previewed, or processed for high-level interpretation. Therefore, on-the-fly correction of such images is important to avoid possible time delays, mitigate computational expenses, and increase image perception quality. We bridge this gap by synthesizing a deconvolution kernel as a linear combination of Finite Impulse Response (FIR) even-derivative filters that can be directly convolved with blurry input images to boost the frequency fall-off of the Point Spread Function (PSF) associated with the optical blur. We employ a Gaussian low-pass filter to decouple the image denoising problem for image edge deblurring. Furthermore, we propose a blind approach to estimate the PSF statistics for two Gaussian and Laplacian models that are common in many imaging pipelines. Thorough experiments are designed to test and validate the efficiency of the proposed method using 2054 naturally blurred images across six imaging applications and seven state-of-the-art deconvolution methods.Comment: 15 pages, for publication in IEEE Transaction Image Processin

Super Resolution of Wavelet-Encoded Images and Videos

In this dissertation, we address the multiframe super resolution reconstruction problem for wavelet-encoded images and videos. The goal of multiframe super resolution is to obtain one or more high resolution images by fusing a sequence of degraded or aliased low resolution images of the same scene. Since the low resolution images may be unaligned, a registration step is required before super resolution reconstruction. Therefore, we first explore in-band (i.e. in the wavelet-domain) image registration; then, investigate super resolution. Our motivation for analyzing the image registration and super resolution problems in the wavelet domain is the growing trend in wavelet-encoded imaging, and wavelet-encoding for image/video compression. Due to drawbacks of widely used discrete cosine transform in image and video compression, a considerable amount of literature is devoted to wavelet-based methods. However, since wavelets are shift-variant, existing methods cannot utilize wavelet subbands efficiently. In order to overcome this drawback, we establish and explore the direct relationship between the subbands under a translational shift, for image registration and super resolution. We then employ our devised in-band methodology, in a motion compensated video compression framework, to demonstrate the effective usage of wavelet subbands. Super resolution can also be used as a post-processing step in video compression in order to decrease the size of the video files to be compressed, with downsampling added as a pre-processing step. Therefore, we present a video compression scheme that utilizes super resolution to reconstruct the high frequency information lost during downsampling. In addition, super resolution is a crucial post-processing step for satellite imagery, due to the fact that it is hard to update imaging devices after a satellite is launched. Thus, we also demonstrate the usage of our devised methods in enhancing resolution of pansharpened multispectral images

Design and Optimization of Graph Transform for Image and Video Compression

The main contribution of this thesis is the introduction of new methods for designing adaptive transforms for image and video compression. Exploiting graph signal processing techniques, we develop new graph construction methods targeted for image and video compression applications. In this way, we obtain a graph that is, at the same time, a good representation of the image and easy to transmit to the decoder. To do so, we investigate different research directions. First, we propose a new method for graph construction that employs innovative edge metrics, quantization and edge prediction techniques. Then, we propose to use a graph learning approach and we introduce a new graph learning algorithm targeted for image compression that defines the connectivities between pixels by taking into consideration the coding of the image signal and the graph topology in rate-distortion term. Moreover, we also present a new superpixel-driven graph transform that uses clusters of superpixel as coding blocks and then computes the graph transform inside each region. In the second part of this work, we exploit graphs to design directional transforms. In fact, an efficient representation of the image directional information is extremely important in order to obtain high performance image and video coding. In this thesis, we present a new directional transform, called Steerable Discrete Cosine Transform (SDCT). This new transform can be obtained by steering the 2D-DCT basis in any chosen direction. Moreover, we can also use more complex steering patterns than a single pure rotation. In order to show the advantages of the SDCT, we present a few image and video compression methods based on this new directional transform. The obtained results show that the SDCT can be efficiently applied to image and video compression and it outperforms the classical DCT and other directional transforms. Along the same lines, we present also a new generalization of the DFT, called Steerable DFT (SDFT). Differently from the SDCT, the SDFT can be defined in one or two dimensions. The 1D-SDFT represents a rotation in the complex plane, instead the 2D-SDFT performs a rotation in the 2D Euclidean space

Wavelet-Based Enhancement Technique for Visibility Improvement of Digital Images

Image enhancement techniques for visibility improvement of color digital images based on wavelet transform domain are investigated in this dissertation research. In this research, a novel, fast and robust wavelet-based dynamic range compression and local contrast enhancement (WDRC) algorithm to improve the visibility of digital images captured under non-uniform lighting conditions has been developed. A wavelet transform is mainly used for dimensionality reduction such that a dynamic range compression with local contrast enhancement algorithm is applied only to the approximation coefficients which are obtained by low-pass filtering and down-sampling the original intensity image. The normalized approximation coefficients are transformed using a hyperbolic sine curve and the contrast enhancement is realized by tuning the magnitude of the each coefficient with respect to surrounding coefficients. The transformed coefficients are then de-normalized to their original range. The detail coefficients are also modified to prevent edge deformation. The inverse wavelet transform is carried out resulting in a lower dynamic range and contrast enhanced intensity image. A color restoration process based on the relationship between spectral bands and the luminance of the original image is applied to convert the enhanced intensity image back to a color image. Although the colors of the enhanced images produced by the proposed algorithm are consistent with the colors of the original image, the proposed algorithm fails to produce color constant results for some pathological scenes that have very strong spectral characteristics in a single band. The linear color restoration process is the main reason for this drawback. Hence, a different approach is required for tackling the color constancy problem. The illuminant is modeled having an effect on the image histogram as a linear shift and adjust the image histogram to discount the illuminant. The WDRC algorithm is then applied with a slight modification, i.e. instead of using a linear color restoration, a non-linear color restoration process employing the spectral context relationships of the original image is applied. The proposed technique solves the color constancy issue and the overall enhancement algorithm provides attractive results improving visibility even for scenes with near-zero visibility conditions. In this research, a new wavelet-based image interpolation technique that can be used for improving the visibility of tiny features in an image is presented. In wavelet domain interpolation techniques, the input image is usually treated as the low-pass filtered subbands of an unknown wavelet-transformed high-resolution (HR) image, and then the unknown high-resolution image is produced by estimating the wavelet coefficients of the high-pass filtered subbands. The same approach is used to obtain an initial estimate of the high-resolution image by zero filling the high-pass filtered subbands. Detail coefficients are estimated via feeding this initial estimate to an undecimated wavelet transform (UWT). Taking an inverse transform after replacing the approximation coefficients of the UWT with initially estimated HR image, results in the final interpolated image. Experimental results of the proposed algorithms proved their superiority over the state-of-the-art enhancement and interpolation techniques

Livrable D3.3 of the PERSEE project : 2D coding tools

49Livrable D3.3 du projet ANR PERSEECe rapport a été réalisé dans le cadre du projet ANR PERSEE (n° ANR-09-BLAN-0170). Exactement il correspond au livrable D3.3 du projet. Son titre : 2D coding tool