Image Restoration with a New Class of Forward-Backward-Forward Diffusion Equations of Perona-Malik Type with Applications to Satellite Image Enhancement

new class of anisotropic diffusion models is proposed for image processing which can be viewed either as a novel kind of regularization of the classical Perona-Malik model or, as advocated by the authors, as a new independent model. The models are diffusive in nature and are characterized by the presence of both forward and backward regimes. In contrast to the Perona-Malik model, in the proposed model the backward regime is confined to a bounded region, and gradients are only allowed to grow up to a large but tunable size, thus effectively preventing indiscriminate singularity formation, i.e., staircasing. Extensive numerical experiments demonstrate that the method is a viable denoising/deblurring tool. The method is significantly faster than competing state-of-the-art methods and appears to be particularly effective for simultaneous denoising and deblurring. An application to satellite image enhancement is also presented.open1

Finding faint HI structure in and around galaxies: scraping the barrel

Soon to be operational HI survey instruments such as APERTIF and ASKAP will produce large datasets. These surveys will provide information about the HI in and around hundreds of galaxies with a typical signal-to-noise ratio of $\sim$ 10 in the inner regions and $\sim$ 1 in the outer regions. In addition, such surveys will make it possible to probe faint HI structures, typically located in the vicinity of galaxies, such as extra-planar-gas, tails and filaments. These structures are crucial for understanding galaxy evolution, particularly when they are studied in relation to the local environment. Our aim is to find optimized kernels for the discovery of faint and morphologically complex HI structures. Therefore, using HI data from a variety of galaxies, we explore state-of-the-art filtering algorithms. We show that the intensity-driven gradient filter, due to its adaptive characteristics, is the optimal choice. In fact, this filter requires only minimal tuning of the input parameters to enhance the signal-to-noise ratio of faint components. In addition, it does not degrade the resolution of the high signal-to-noise component of a source. The filtering process must be fast and be embedded in an interactive visualization tool in order to support fast inspection of a large number of sources. To achieve such interactive exploration, we implemented a multi-core CPU (OpenMP) and a GPU (OpenGL) version of this filter in a 3D visualization environment ($\tt{SlicerAstro}$).Comment: 17 pages, 9 figures, 4 tables. Astronomy and Computing, accepte

Eigenvector Model Descriptors for Solving an Inverse Problem of Helmholtz Equation: Extended Materials

We study the seismic inverse problem for the recovery of subsurface properties in acoustic media. In order to reduce the ill-posedness of the problem, the heterogeneous wave speed parameter to be recovered is represented using a limited number of coefficients associated with a basis of eigenvectors of a diffusion equation, following the regularization by discretization approach. We compare several choices for the diffusion coefficient in the partial differential equations, which are extracted from the field of image processing. We first investigate their efficiency for image decomposition (accuracy of the representation with respect to the number of variables and denoising). Next, we implement the method in the quantitative reconstruction procedure for seismic imaging, following the Full Waveform Inversion method, where the difficulty resides in that the basis is defined from an initial model where none of the actual structures is known. In particular, we demonstrate that the method is efficient for the challenging reconstruction of media with salt-domes. We employ the method in two and three-dimensional experiments and show that the eigenvector representation compensates for the lack of low frequency information, it eventually serves us to extract guidelines for the implementation of the method.Comment: 45 pages, 37 figure

Connections Between Numerical Algorithms for PDEs and Neural Networks

We investigate numerous structural connections between numerical algorithms for partial differential equations (PDEs) and neural architectures. Our goal is to transfer the rich set of mathematical foundations from the world of PDEs to neural networks. Besides structural insights, we provide concrete examples and experimental evaluations of the resulting architectures. Using the example of generalised nonlinear diffusion in 1D, we consider explicit schemes, acceleration strategies thereof, implicit schemes, and multigrid approaches. We connect these concepts to residual networks, recurrent neural networks, and U-net architectures. Our findings inspire a symmetric residual network design with provable stability guarantees and justify the effectiveness of skip connections in neural networks from a numerical perspective. Moreover, we present U-net architectures that implement multigrid techniques for learning efficient solutions of partial differential equation models, and motivate uncommon design choices such as trainable nonmonotone activation functions. Experimental evaluations show that the proposed architectures save half of the trainable parameters and can thus outperform standard ones with the same model complexity. Our considerations serve as a basis for explaining the success of popular neural architectures and provide a blueprint for developing new mathematically well-founded neural building blocks

Refined upper bounds on the coarsening rate of discrete, ill-posed diffusion equations

"We study coarsening phenomena observed in discrete, ill-posed diffusion equations that arise in a variety of applications, including computer vision, population dynamics and granular flow. Our results provide rigorous upper bounds on the coarsening rate in any dimension. Heuristic arguments and the numerical experiments we perform indicate that the bounds are in agreement with the actual rate of coarsening."http://deepblue.lib.umich.edu/bitstream/2027.42/64211/1/non8_12_002.pd

Nonlinear diffusion filtering on surfaces

Nonlinear diffusion filtering is a PDE-based method to remove noise from images that has found much success. This dissertation looks at whether nonlinear diffusion filtering can be combined with the closest point method, a relatively new and novel method for solving partial differential equations on surfaces. The closest point method is an embedding method that uses a simple representation of surfaces. The theory and implementation of for the closest point method is presented. We perform convergence studies that show good agreement with theory.\ud \ud We discuss the use of linear and nonlinear diffusion for image processing, in particular the Perona{Malik and Gaussian schemes. We show that they can be combined with the closest point method to produce impressive results, visualised beautifully using an OpenGL raytracer designed for use with the closest point method. Some surprising and unexpected eects were discovered when moving from a plane to a three-dimensional surface. These effects are described and investigated

Connecting mathematical models for image processing and neural networks

This thesis deals with the connections between mathematical models for image processing and deep learning. While data-driven deep learning models such as neural networks are flexible and well performing, they are often used as a black box. This makes it hard to provide theoretical model guarantees and scientific insights. On the other hand, more traditional, model-driven approaches such as diffusion, wavelet shrinkage, and variational models offer a rich set of mathematical foundations. Our goal is to transfer these foundations to neural networks. To this end, we pursue three strategies. First, we design trainable variants of traditional models and reduce their parameter set after training to obtain transparent and adaptive models. Moreover, we investigate the architectural design of numerical solvers for partial differential equations and translate them into building blocks of popular neural network architectures. This yields criteria for stable networks and inspires novel design concepts. Lastly, we present novel hybrid models for inpainting that rely on our theoretical findings. These strategies provide three ways for combining the best of the two worlds of model- and data-driven approaches. Our work contributes to the overarching goal of closing the gap between these worlds that still exists in performance and understanding.Gegenstand dieser Arbeit sind die Zusammenhänge zwischen mathematischen Modellen zur Bildverarbeitung und Deep Learning. Während datengetriebene Modelle des Deep Learning wie z.B. neuronale Netze flexibel sind und gute Ergebnisse liefern, werden sie oft als Black Box eingesetzt. Das macht es schwierig, theoretische Modellgarantien zu liefern und wissenschaftliche Erkenntnisse zu gewinnen. Im Gegensatz dazu bieten traditionellere, modellgetriebene Ansätze wie Diffusion, Wavelet Shrinkage und Variationsansätze eine Fülle von mathematischen Grundlagen. Unser Ziel ist es, diese auf neuronale Netze zu übertragen. Zu diesem Zweck verfolgen wir drei Strategien. Zunächst entwerfen wir trainierbare Varianten von traditionellen Modellen und reduzieren ihren Parametersatz, um transparente und adaptive Modelle zu erhalten. Außerdem untersuchen wir die Architekturen von numerischen Lösern für partielle Differentialgleichungen und übersetzen sie in Bausteine von populären neuronalen Netzwerken. Daraus ergeben sich Kriterien für stabile Netzwerke und neue Designkonzepte. Schließlich präsentieren wir neuartige hybride Modelle für Inpainting, die auf unseren theoretischen Erkenntnissen beruhen. Diese Strategien bieten drei Möglichkeiten, das Beste aus den beiden Welten der modell- und datengetriebenen Ansätzen zu vereinen. Diese Arbeit liefert einen Beitrag zum übergeordneten Ziel, die Lücke zwischen den zwei Welten zu schließen, die noch in Bezug auf Leistung und Modellverständnis besteht.ERC Advanced Grant INCOVI

Upper bounds on the coarsening rate of discrete, ill-posed nonlinear diffusion equations

We prove a weak upper bound on the coarsening rate of the discrete-in-space version of an ill-posed, nonlinear diffusion equation. The continuum version of the equation violates parabolicity and lacks a complete well-posedness theory. In particular, numerical simulations indicate very sensitive dependence on initial data. Nevertheless, models based on its discrete-in-space version, which we study, are widely used in a number of applications, including population dynamics (chemotactic movement of bacteria), granular flow (formation of shear bands), and computer vision (image denoising and segmentation). Our bounds have implications for all three applications. © 2008 Wiley Periodicals, Inc.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/61227/1/20259_ftp.pd

A discrete graph Laplacian for signal processing

In this thesis we exploit diffusion processes on graphs to effect two fundamental problems of image processing: denoising and segmentation. We treat these two low-level vision problems on the pixel-wise level under a unified framework: a graph embedding. Using this framework opens us up to the possibilities of exploiting recently introduced algorithms from the semi-supervised machine learning literature. We contribute two novel edge-preserving smoothing algorithms to the literature. Furthermore we apply these edge-preserving smoothing algorithms to some computational photography tasks. Many recent computational photography tasks require the decomposition of an image into a smooth base layer containing large scale intensity variations and a residual layer capturing fine details. Edge-preserving smoothing is the main computational mechanism in producing these multi-scale image representations. We, in effect, introduce a new approach to edge-preserving multi-scale image decompositions. Where as prior approaches such as the Bilateral filter and weighted-least squares methods require multiple parameters to tune the response of the filters our method only requires one. This parameter can be interpreted as a scale parameter. We demonstrate the utility of our approach by applying the method to computational photography tasks that utilise multi-scale image decompositions. With minimal modification to these edge-preserving smoothing algorithms we show that we can extend them to produce interactive image segmentation. As a result the operations of segmentation and denoising are conducted under a unified framework. Moreover we discuss how our method is related to region based active contours. We benchmark our proposed interactive segmentation algorithms against those based upon energy-minimisation, specifically graph-cut methods. We demonstrate that we achieve competitive performance