Entropy Based Cartoon Texture Separation

Separating an image into cartoon and texture components comes useful in image processing applications, such as image compression, image segmentation, image inpainting. Yves Meyer's influential cartoon texture decomposition model involves deriving an energy functional by choosing appropriate spaces and functionals. Minimizers of the derived energy functional are cartoon and texture components of an image. In this study, cartoon part of an image is separated, by reconstructing it from pixels of multi scale Total-Variation filtered versions of the original image which is sought to be decomposed into cartoon and texture parts. An information theoretic pixel by pixel selection criteria is employed to choose the contributing pixels and their scales.Comment: 12 page

A Dual Alternating Direction Method of Multipliers for Image Decomposition and Restoration

In this paper, we develop a dual alternating direction method of multipliers (ADMM) for an image decomposition model. In this model, an image is divided into two meaningful components, i.e., a cartoon part and a texture part. The optimization algorithm that we develop not only gives the cartoon part and the texture part of an image but also gives the restored image (cartoon part + texture part). We also present the global convergence and the local linear convergence rate for the algorithm under some mild conditions. Numerical experiments demonstrate the efficiency and robustness of the dual ADMM (dADMM). Furthermore, we can obtain relatively higher signalto-noise ratio (SNR) comparing to other algorithms. It shows that the choice of the algorithm is also important even for the same model.Comment: 12 pages, 13 figure

Segmentation of Scanning Tunneling Microscopy Images Using Variational Methods and Empirical Wavelets

In the fields of nanoscience and nanotechnology, it is important to be able to functionalize surfaces chemically for a wide variety of applications. Scanning tunneling microscopes (STMs) are important instruments in this area used to measure the surface structure and chemistry with better than molecular resolution. Self-assembly is frequently used to create monolayers that redefine the surface chemistry in just a single-molecule-thick layer. Indeed, STM images reveal rich information about the structure of self-assembled monolayers since they convey chemical and physical properties of the studied material. In order to assist in and to enhance the analysis of STM and other images, we propose and demonstrate an image-processing framework that produces two image segmentations: one is based on intensities (apparent heights in STM images) and the other is based on textural patterns. The proposed framework begins with a cartoon+texture decomposition, which separates an image into its cartoon and texture components. Afterward, the cartoon image is segmented by a modified multiphase version of the local Chan-Vese model, while the texture image is segmented by a combination of 2D empirical wavelet transform and a clustering algorithm. Overall, our proposed framework contains several new features, specifically in presenting a new application of cartoon+texture decomposition and of the empirical wavelet transforms and in developing a specialized framework to segment STM images and other data. To demonstrate the potential of our approach, we apply it to actual STM images of cyanide monolayers on Au\{111\} and present their corresponding segmentation results

Image Cartoon-Texture Decomposition Using Isotropic Patch Recurrence

Aiming at separating the cartoon and texture layers from an image, cartoon-texture decomposition approaches resort to image priors to model cartoon and texture respectively. In recent years, patch recurrence has emerged as a powerful prior for image recovery. However, the existing strategies of using patch recurrence are ineffective to cartoon-texture decomposition, as both cartoon contours and texture patterns exhibit strong patch recurrence in images. To address this issue, we introduce the isotropy prior of patch recurrence, that the spatial configuration of similar patches in texture exhibits the isotropic structure which is different from that in cartoon, to model the texture component. Based on the isotropic patch recurrence, we construct a nonlocal sparsification system which can effectively distinguish well-patterned features from contour edges. Incorporating the constructed nonlocal system into morphology component analysis, we develop an effective method to both noiseless and noisy cartoon-texture decomposition. The experimental results have demonstrated the superior performance of the proposed method to the existing ones, as well as the effectiveness of the isotropic patch recurrence prior.Comment: 13 pages, 10 figure

A Study of Cross-domain Generative Models applied to Cartoon Series

We investigate Generative Adversarial Networks (GANs) to model one particular kind of image: frames from TV cartoons. Cartoons are particularly interesting because their visual appearance emphasizes the important semantic information about a scene while abstracting out the less important details, but each cartoon series has a distinctive artistic style that performs this abstraction in different ways. We consider a dataset consisting of images from two popular television cartoon series, Family Guy and The Simpsons. We examine the ability of GANs to generate images from each of these two domains, when trained independently as well as on both domains jointly. We find that generative models may be capable of finding semantic-level correspondences between these two image domains despite the unsupervised setting, even when the training data does not give labeled alignments between them

Multi-model full-waveform inversion

We propose a multi-model formulation of full-waveform inversion that is similar to image decomposition into a "cartoon" and "texture" used in image processing. Inversion problem is formulated as unconstrained multi-norm optimization that can be solved using conventional iterative solvers. We demonstrate the proposed model decomposition approach by recovering a blocky subsurface seismic model from noisy data in time-lapse and single-model full-waveform inversion problems.Comment: 6 pages, 6 figure

Clustered Sparsity and Separation of Cartoon and Texture

Natural images are typically a composition of cartoon and texture structures. A medical image might, for instance, show a mixture of gray matter and the skull cap. One common task is to separate such an image into two single images, one containing the cartoon part and the other containing the texture part. Recently, a powerful class of algorithms using sparse approximation and $\ell_1$ minimization has been introduced to resolve this problem, and numerous inspiring empirical results have already been obtained. In this paper we provide the first thorough theoretical study of the separation of a combination of cartoon and texture structures in a model situation using this class of algorithms. The methodology we consider expands the image in a combined dictionary consisting of a curvelet tight frame and a Gabor tight frame and minimizes the $\ell_1$ norm on the analysis side. Sparse approximation properties then force the cartoon components into the curvelet coefficients and the texture components into the Gabor coefficients, thereby separating the image. Utilizing the fact that the coefficients are clustered geometrically, we prove that at sufficiently fine scales arbitrarily precise separation is possible. Main ingredients of our analysis are the novel notion of cluster coherence and clustered/geometric sparsity. Our analysis also provides a deep understanding on when separation is still possible.Comment: 25 pages, 6 figure

Generating a Fusion Image: One's Identity and Another's Shape

Generating a novel image by manipulating two input images is an interesting research problem in the study of generative adversarial networks (GANs). We propose a new GAN-based network that generates a fusion image with the identity of input image x and the shape of input image y. Our network can simultaneously train on more than two image datasets in an unsupervised manner. We define an identity loss LI to catch the identity of image x and a shape loss LS to get the shape of y. In addition, we propose a novel training method called Min-Patch training to focus the generator on crucial parts of an image, rather than its entirety. We show qualitative results on the VGG Youtube Pose dataset, Eye dataset (MPIIGaze and UnityEyes), and the Photo-Sketch-Cartoon dataset.Comment: To appear in CVPR 201

DragonPaint: Rule based bootstrapping for small data with an application to cartoon coloring

In this paper, we confront the problem of deep learning's big labeled data requirements, offer a rule based strategy for extreme augmentation of small data sets and apply that strategy with the image to image translation model by Isola et al. (2016) to automate cel style cartoon coloring with very limited training data. While our experimental results using geometric rules and transformations demonstrate the performance of our methods on an image translation task with industry applications in art, design and animation, we also propose the use of rules on partial data sets as a generalizable small data strategy, potentially applicable across data types and domains

Simultaneous Inpainting and Denoising by Directional Global Three-part Decomposition: Connecting Variational and Fourier Domain Based Image Processing

We consider the very challenging task of restoring images (i) which have a large number of missing pixels, (ii) whose existing pixels are corrupted by noise and (iii) the ideal image to be restored contains both cartoon and texture elements. The combination of these three properties makes this inverse problem a very difficult one. The solution proposed in this manuscript is based on directional global three-part decomposition (DG3PD) [ThaiGottschlich2016] with directional total variation norm, directional G-norm and $\ell_\infty$-norm in curvelet domain as key ingredients of the model. Image decomposition by DG3PD enables a decoupled inpainting and denoising of the cartoon and texture components. A comparison to existing approaches for inpainting and denoising shows the advantages of the proposed method. Moreover, we regard the image restoration problem from the viewpoint of a Bayesian framework and we discuss the connections between the proposed solution by function space and related image representation by harmonic analysis and pyramid decomposition