Scale-aware decomposition of images based on patch-based filtering

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2015. 2. 조남익.This dissertation presents an image decomposition algorithm based on patch-based filtering, for splitting an image into a structure layer and a texture layer. There are many applications through the decomposition because each layer can be processed respectively and appropriate manipulations are accomplished. Generally, structure layer captures coarse structure with large discontinuities and a texture layer contains fine details or proper patterns. The image decomposition is done by edge-preserving smoothing where structure layer can be obtained by applying smoothing filters to an image and then a texture layer by subtracting the filtered image from the original. The main contribution of this dissertation is to design an efficient and effective edge-preserving filter that can be adapted to various scales of images. The advantage of the proposed decomposition scheme is that it is robust to noise and can be extended to a noisy image decomposition, while conventional image decomposition methods cannot be applied to a noisy image decomposition and conventional image denoising methods are not suitable for image decomposition. To be specific, a patch-based framework is proposed in this dissertation, which is efficient in image denoising and it is designed to smooth an image while preserving details and texture. Specifically, given a pixel, the filtering output is computed as the weighted average of neighboring pixels. For computing the weights, a set of similar patches is found at each pixel by considering patch similarities based on mean squared error (MSE) and other constraints. Then, weights between each patch and its similar patches are computed respectively. With the patch weights, all the pixels in a patch are updated at the same time while adapting to the local pixel weight. For better edge-preserving smoothing, the proposed algorithm utilizes two iterations which are performed through the same smoothing filter with different parameters. Also kernel bandwidth and the number of similar patches are tuned for multi-scale image decomposition. The proposed decomposition can be applied to many applications, such as HDR tone mapping, detail enhancement, image denoising, and image coding, etc. In detail enhancement, the proposed smoothing filter is utilized to extract image detail and enhance it. In HDR tone mapping, a typical framework is used where the smoothing operator is replaced by the proposed one to reduce contrast range of a high dynamic range image to display it on low dynamic range devices. For image denoising, a noisy input is decomposed into structure/texture/noise and the noise layer is discarded while the texture layer is restored through the histogram matching. Also a novel coding scheme named as ``structure scalable image coding scheme'' is proposed where structure layer and salient texture layer are encoded for efficient image coding. Experimental results show that the proposed framework works well for image decomposition and it is robust to the presence of noise. Also it is verified that the proposed work can be utilized in many applications. In addition, by adopting the proposed method in decomposition of a noisy image, both image denoising and image enhancement can be achieved in the proposed framework. Furthermore, the proposed image coding method reduces compression artifact and improve the performance of image coding.Abstract i Contents iv List of Figures vi List of Tables xi 1 Introduction 1 1.1 Image decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Image enhancement . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Image denoising . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3.1 Spatial denoising . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3.2 Transformdomain denoising . . . . . . . . . . . . . . . . . . 9 1.3.3 benefits of combined image decomposition and image denoising 9 1.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2 Related work 17 2.1 Image decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.1.1 Laplacian subbands . . . . . . . . . . . . . . . . . . . . . . . 17 2.2 Edge-preserving smoothing . . . . . . . . . . . . . . . . . . . . . . . 18 2.2.1 Bilateral filtering . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.2.2 Nonlocal means filtering . . . . . . . . . . . . . . . . . . . . . 21 3 Scale-aware decomposition of images based on patch-based filtering 23 3.1 Edge-preserving smoothing via patch-based framework . . . . . . . . 23 3.2 Multi-scale image decomposition . . . . . . . . . . . . . . . . . . . . 26 4 Applications 31 4.1 Image enhancement . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.1.1 Detail enhancement . . . . . . . . . . . . . . . . . . . . . . . 31 4.1.2 HDR tone mapping . . . . . . . . . . . . . . . . . . . . . . . 36 4.2 Image denoising . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.2.1 A noisy image decomposition . . . . . . . . . . . . . . . . . . 40 4.2.2 texture enhancement via histogram preservation . . . . . . . 41 4.2.3 image denoising via subband BLF . . . . . . . . . . . . . . . 44 4.2.4 Experimental results of image denoising . . . . . . . . . . . . 48 4.3 Image coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.3.1 Structure scalable image coding framework . . . . . . . . . . 61 5 Conclusion 73 Bibliography 76Docto

Multi-Scale Edge Detection Algorithms and Their Information-Theoretic Analysis in the Context of Visual Communication

The unrealistic assumption that noise can be modeled as independent, additive and uniform can lead to problems when edge detection methods are applied to low signal-to-noise ratio (SNR) images. The main reason for this is because the filter scale and the threshold for the gradient are difficult to determine at a regional or local scale when the noise estimate is on a global scale. Therefore, in this dissertation, we attempt to solve these problems by using more than one filter to detect the edges and discarding the global thresholding method in the edge discrimination. The proposed multi-scale edge detection algorithms utilize the multi-scale description to detect and localize edges. Furthermore, instead of using the single default global threshold, a local dynamic threshold is introduced to discriminate between edges and non-edges. The proposed algorithms also perform connectivity analysis on edge maps to ensure that small, disconnected edges are removed. Experiments where the methods are applied to a sequence of images of the same scene with different SNRs show the methods to be robust to noise. Additionally, a new noise reduction algorithm based on the multi-scale edge analysis is proposed. In general, an edge—high frequency information in an image—would be filtered or suppressed after image smoothing. With the help of multi-scale edge detection algorithms, the overall edge structure of the original image could be preserved when only the isolated edge information that represents noise gets filtered out. Experimental results show that this method is robust to high levels of noise, correctly preserving the edges. We also propose a new method for evaluating the performance of edge detection algorithms. It is based on information-theoretic analysis of the edge detection algorithms in the context of an end-to-end visual communication channel. We use the information between the scene and the output of the edge-detection algorithm, ala Shannon, to evaluate the performance. An edge detection algorithm is considered to have high performance only if the information rate from the scene to the edge approaches the maximum possible. Therefore, this information-theoretic analysis becomes a new method to allow comparison between different edge detection operators for a given end-to-end image processing system

Outlier robust corner-preserving methods for reconstructing noisy images

The ability to remove a large amount of noise and the ability to preserve most structure are desirable properties of an image smoother. Unfortunately, they usually seem to be at odds with each other; one can only improve one property at the cost of the other. By combining M-smoothing and least-squares-trimming, the TM-smoother is introduced as a means to unify corner-preserving properties and outlier robustness. To identify edge- and corner-preserving properties, a new theory based on differential geometry is developed. Further, robustness concepts are transferred to image processing. In two examples, the TM-smoother outperforms other corner-preserving smoothers. A software package containing both the TM- and the M-smoother can be downloaded from the Internet.Comment: Published at http://dx.doi.org/10.1214/009053606000001109 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Image Processing Techniques for Assessing Contractility in Isolated Adult Cardiac Myocytes

We describe a computational framework for the comprehensive assessment of contractile responses of enzymatically dissociated adult cardiac myocytes. The proposed methodology comprises the following stages: digital video recording of the contracting cell, edge preserving total variation-based image smoothing, segmentation of the smoothed images, contour extraction from the segmented images, shape representation by Fourier descriptors, and contractility assessment. The different stages are variants of mathematically sound and computationally robust algorithms very well established in the image processing community. The physiologic application of the methodology is evaluated by assessing overall contraction in enzymatically dissociated adult rat cardiocytes. Our results demonstrate the effectiveness of the proposed approach in characterizing the true, two-dimensional, “shortening” in the contraction process of adult cardiocytes. We compare the performance of the proposed method to that of a popular edge detection system in the literature. The proposed method not only provides a more comprehensive assessment of the myocyte contraction process but also can potentially eliminate historical concerns and sources of errors caused by myocyte rotation or translation during contraction. Furthermore, the versatility of the image processing techniques makes the method suitable for determining myocyte shortening in cells that usually bend or move during contraction. The proposed method can be utilized to evaluate changes in contractile behavior resulting from drug intervention, disease modeling, transgeneity, or other common applications to mammalian cardiocytes

A Convex Model for Edge-Histogram Specification with Applications to Edge-preserving Smoothing

The goal of edge-histogram specification is to find an image whose edge image has a histogram that matches a given edge-histogram as much as possible. Mignotte has proposed a non-convex model for the problem [M. Mignotte. An energy-based model for the image edge-histogram specification problem. IEEE Transactions on Image Processing, 21(1):379--386, 2012]. In his work, edge magnitudes of an input image are first modified by histogram specification to match the given edge-histogram. Then, a non-convex model is minimized to find an output image whose edge-histogram matches the modified edge-histogram. The non-convexity of the model hinders the computations and the inclusion of useful constraints such as the dynamic range constraint. In this paper, instead of considering edge magnitudes, we directly consider the image gradients and propose a convex model based on them. Furthermore, we include additional constraints in our model based on different applications. The convexity of our model allows us to compute the output image efficiently using either Alternating Direction Method of Multipliers or Fast Iterative Shrinkage-Thresholding Algorithm. We consider several applications in edge-preserving smoothing including image abstraction, edge extraction, details exaggeration, and documents scan-through removal. Numerical results are given to illustrate that our method successfully produces decent results efficiently

A Brief Survey of Recent Edge-Preserving Smoothers

We introduce recent and very recent smoothing methods and discuss them in the common framework of `energy functions'. Focus is on the preservation of boundaries, spikes and canyons in presence of noise