A fully conservative parallel numerical algorithm with adaptive spatial grid for solving nonlinear diffusion equations in image processing

In this paper we present simple yet efficient parallel program implementation of grid-difference method for solving nonlinear parabolic equations, which satisfies both fully conservative property and second order of approximation on non-uniform spatial grid according to geometrical sanity of a task. The proposed algorithm was tested on Perona–Malik method for image noise ltering task based on differential equations. Also in this work we propose generalization of the Perona–Malik equation, which is a one of diffusion in complex-valued region type. This corresponds to the conversion to such types of nonlinear equations like Leontovich–Fock equation with a dependent on the gradient field according to the nonlinear law coefficient of diffraction. This is a special case of generalization of the Perona–Malik equation to the multicomponent case. This approach makes noise removal process more flexible by increasing its capabilities, which allows achieving better results for the task of image denoising

Connecting mathematical models for image processing and neural networks

This thesis deals with the connections between mathematical models for image processing and deep learning. While data-driven deep learning models such as neural networks are flexible and well performing, they are often used as a black box. This makes it hard to provide theoretical model guarantees and scientific insights. On the other hand, more traditional, model-driven approaches such as diffusion, wavelet shrinkage, and variational models offer a rich set of mathematical foundations. Our goal is to transfer these foundations to neural networks. To this end, we pursue three strategies. First, we design trainable variants of traditional models and reduce their parameter set after training to obtain transparent and adaptive models. Moreover, we investigate the architectural design of numerical solvers for partial differential equations and translate them into building blocks of popular neural network architectures. This yields criteria for stable networks and inspires novel design concepts. Lastly, we present novel hybrid models for inpainting that rely on our theoretical findings. These strategies provide three ways for combining the best of the two worlds of model- and data-driven approaches. Our work contributes to the overarching goal of closing the gap between these worlds that still exists in performance and understanding.Gegenstand dieser Arbeit sind die Zusammenhänge zwischen mathematischen Modellen zur Bildverarbeitung und Deep Learning. Während datengetriebene Modelle des Deep Learning wie z.B. neuronale Netze flexibel sind und gute Ergebnisse liefern, werden sie oft als Black Box eingesetzt. Das macht es schwierig, theoretische Modellgarantien zu liefern und wissenschaftliche Erkenntnisse zu gewinnen. Im Gegensatz dazu bieten traditionellere, modellgetriebene Ansätze wie Diffusion, Wavelet Shrinkage und Variationsansätze eine Fülle von mathematischen Grundlagen. Unser Ziel ist es, diese auf neuronale Netze zu übertragen. Zu diesem Zweck verfolgen wir drei Strategien. Zunächst entwerfen wir trainierbare Varianten von traditionellen Modellen und reduzieren ihren Parametersatz, um transparente und adaptive Modelle zu erhalten. Außerdem untersuchen wir die Architekturen von numerischen Lösern für partielle Differentialgleichungen und übersetzen sie in Bausteine von populären neuronalen Netzwerken. Daraus ergeben sich Kriterien für stabile Netzwerke und neue Designkonzepte. Schließlich präsentieren wir neuartige hybride Modelle für Inpainting, die auf unseren theoretischen Erkenntnissen beruhen. Diese Strategien bieten drei Möglichkeiten, das Beste aus den beiden Welten der modell- und datengetriebenen Ansätzen zu vereinen. Diese Arbeit liefert einen Beitrag zum übergeordneten Ziel, die Lücke zwischen den zwei Welten zu schließen, die noch in Bezug auf Leistung und Modellverständnis besteht.ERC Advanced Grant INCOVI

On the application of partial differential equations and fractional partial differential equations to images and their methods of solution

This body of work examines the plausibility of applying partial di erential equations and time-fractional partial di erential equations to images. The standard di usion equation is coupled with a nonlinear cubic source term of the Fitzhugh-Nagumo type to obtain a model with di usive properties and a binarizing e ect due to the source term. We examine the e ects of applying this model to a class of images known as document images; images that largely comprise text. The e ects of this model result in a binarization process that is competitive with the state-of-the-art techniques. Further to this application, we provide a stability analysis of the method as well as high-performance implementation on general purpose graphical processing units. The model is extended to include time derivatives to a fractional order which a ords us another degree of control over this process and the nature of the fractionality is discussed indicating the change in dynamics brought about by this generalization. We apply a semi-discrete method derived by hybridizing the Laplace transform and two discretization methods: nite-di erences and Chebyshev collocation. These hybrid techniques are coupled with a quasi-linearization process to allow for the application of the Laplace transform, a linear operator, to a nonlinear equation of fractional order in the temporal domain. A thorough analysis of these methods is provided giving rise to conditions for solvability. The merits and demerits of the methods are discussed indicating the appropriateness of each method

A fourth-order PDE denoising model with an adaptive relaxation method

In this paper, an adaptive relaxation method and a discontinuity treatment of edges are proposed to improve the digital image denoising process by using the fourth-order partial differential equation (known as the YK model) first proposed by You and Kaveh. Since the YK model would generate some speckles into the denoised image, a relaxation method is incorporated into the model to reduce the formation of isolated speckles. An additional improvement is employed to handle the discontinuity on the edges of the image. In order to stop the iteration automatically, a control of the iteration is integrated into the denoising process. Numerical results demonstrate that such modifications not only make the denoised image look more natural, but also achieve a higher value of PSNR

Fast wide-volume functional imaging of engineered in vitro brain tissues

The need for in vitro models that mimic the human brain to replace animal testing and allow high-throughput screening has driven scientists to develop new tools that reproduce tissue-like features on a chip. Three-dimensional (3D) in vitro cultures are emerging as an unmatched platform that preserves the complexity of cell-to-cell connections within a tissue, improves cell survival, and boosts neuronal differentiation. In this context, new and flexible imaging approaches are required to monitor the functional states of 3D networks. Herein, we propose an experimental model based on 3D neuronal networks in an alginate hydrogel, a tunable wide-volume imaging approach, and an efficient denoising algorithm to resolve, down to single cell resolution, the 3D activity of hundreds of neurons expressing the calcium sensor GCaMP6s. Furthermore, we implemented a 3D co-culture system mimicking the contiguous interfaces of distinct brain tissues such as the cortical-hippocampal interface. The analysis of the network activity of single and layered neuronal co-cultures revealed cell-type-specific activities and an organization of neuronal subpopulations that changed in the two culture configurations. Overall, our experimental platform represents a simple, powerful and cost-effective platform for developing and monitoring living 3D layered brain tissue on chip structures with high resolution and high throughput

A volume filtering and rendering system for an improved visual balance of feature preservation and noise suppression in medical imaging

Preserving or enhancing salient features whilst effectively suppressing noise-derived artifacts and extraneous detail have been two consistent yet competing objectives in volumetric medical image processing. Illustrative techniques (and methods inspired by them) can help to enhance and, if desired, isolate the depiction of specific regions of interest whilst retaining overall context. However, highlighting or enhancing specific features can have the undesirable side-effect of highlighting noise. Second-derivative based methods can be employed effectively in both the rendering and volume filtering stages of a visualisation pipeline to enhance the depiction of feature detail whilst minimising noise-based artifacts. We develop a new 3D anisotropic-diffusion PDE for an improved balance of feature-retention and noise reduction; furthermore, we present a feature-enhancing visualisation pipeline that can be applied to multiple modalities and has been shown to be particularly effective in the context of 3D ultrasound

Integrating IoT and Novel Approaches to Enhance Electromagnetic Image Quality using Modern Anisotropic Diffusion and Speckle Noise Reduction Techniques

Electromagnetic imaging is becoming more important in many sectors, and this requires high-quality pictures for reliable analysis. This study makes use of the complementary relationship between IoT and current image processing methods to improve the quality of electromagnetic images. The research presents a new framework for connecting Internet of Things sensors to imaging equipment, allowing for instantaneous input and adjustment. At the same time, the suggested system makes use of sophisticated anisotropic diffusion algorithms to bring out key details and hide noise in electromagnetic pictures. In addition, a cutting-edge technique for reducing speckle noise is used to combat this persistent issue in electromagnetic imaging. The effectiveness of the suggested system was determined via a comparison to standard imaging techniques. There was a noticeable improvement in visual sharpness, contrast, and overall clarity without any loss of information, as shown by the results. Incorporating IoT sensors also facilitated faster calibration and real-time modifications, which opened up new possibilities for use in contexts with a high degree of variation. In fields where electromagnetic imaging plays a crucial role, such as medicine, remote sensing, and aerospace, the ramifications of this study are far-reaching. Our research demonstrates how the Internet of Things (IoT) and cutting-edge image processing have the potential to dramatically improve the functionality and versatility of electromagnetic imaging systems

The jump set under geometric regularisation. Part 1: Basic technique and first-order denoising

Let u \in \mbox{BV}(\Omega) solve the total variation denoising problem with $L^2$-squared fidelity and data $f$. Caselles et al. [Multiscale Model. Simul. 6 (2008), 879--894] have shown the containment $\mathcal{H}^{m-1}(J_u \setminus J_f)=0$ of the jump set $J_u$ of $u$ in that of $f$. Their proof unfortunately depends heavily on the co-area formula, as do many results in this area, and as such is not directly extensible to higher-order, curvature-based, and other advanced geometric regularisers, such as total generalised variation (TGV) and Euler's elastica. These have received increased attention in recent times due to their better practical regularisation properties compared to conventional total variation or wavelets. We prove analogous jump set containment properties for a general class of regularisers. We do this with novel Lipschitz transformation techniques, and do not require the co-area formula. In the present Part 1 we demonstrate the general technique on first-order regularisers, while in Part 2 we will extend it to higher-order regularisers. In particular, we concentrate in this part on TV and, as a novelty, Huber-regularised TV. We also demonstrate that the technique would apply to non-convex TV models as well as the Perona-Malik anisotropic diffusion, if these approaches were well-posed to begin with

Understanding and advancing PDE-based image compression

This thesis is dedicated to image compression with partial differential equations (PDEs). PDE-based codecs store only a small amount of image points and propagate their information into the unknown image areas during the decompression step. For certain classes of images, PDE-based compression can already outperform the current quasi-standard, JPEG2000. However, the reasons for this success are not yet fully understood, and PDE-based compression is still in a proof-of-concept stage. With a probabilistic justification for anisotropic diffusion, we contribute to a deeper insight into design principles for PDE-based codecs. Moreover, by analysing the interaction between efficient storage methods and image reconstruction with diffusion, we can rank PDEs according to their practical value in compression. Based on these observations, we advance PDE-based compression towards practical viability: First, we present a new hybrid codec that combines PDE- and patch-based interpolation to deal with highly textured images. Furthermore, a new video player demonstrates the real-time capacities of PDE-based image interpolation and a new region of interest coding algorithm represents important image areas with high accuracy. Finally, we propose a new framework for diffusion-based image colourisation that we use to build an efficient codec for colour images. Experiments on real world image databases show that our new method is qualitatively competitive to current state-of-the-art codecs.Diese Dissertation ist der Bildkompression mit partiellen Differentialgleichungen (PDEs, partial differential equations) gewidmet. PDE-Codecs speichern nur einen geringen Anteil aller Bildpunkte und transportieren deren Information in fehlende Bildregionen. In einigen Fällen kann PDE-basierte Kompression den aktuellen Quasi-Standard, JPEG2000, bereits schlagen. Allerdings sind die Gründe für diesen Erfolg noch nicht vollständig erforscht, und PDE-basierte Kompression befindet sich derzeit noch im Anfangsstadium. Wir tragen durch eine probabilistische Rechtfertigung anisotroper Diffusion zu einem tieferen Verständnis PDE-basierten Codec-Designs bei. Eine Analyse der Interaktion zwischen effizienten Speicherverfahren und Bildrekonstruktion erlaubt es uns, PDEs nach ihrem Nutzen für die Kompression zu beurteilen. Anhand dieser Einsichten entwickeln wir PDE-basierte Kompression hinsichtlich ihrer praktischen Nutzbarkeit weiter: Wir stellen einen Hybrid-Codec für hochtexturierte Bilder vor, der umgebungsbasierte Interpolation mit PDEs kombiniert. Ein neuer Video-Dekodierer demonstriert die Echtzeitfähigkeit PDE-basierter Interpolation und eine Region-of-Interest-Methode erlaubt es, wichtige Bildbereiche mit hoher Genauigkeit zu speichern. Schlussendlich stellen wir ein neues diffusionsbasiertes Kolorierungsverfahren vor, welches uns effiziente Kompression von Farbbildern ermöglicht. Experimente auf Realwelt-Bilddatenbanken zeigen die Konkurrenzfähigkeit dieses Verfahrens auf

Understanding and advancing PDE-based image compression

This thesis is dedicated to image compression with partial differential equations (PDEs). PDE-based codecs store only a small amount of image points and propagate their information into the unknown image areas during the decompression step. For certain classes of images, PDE-based compression can already outperform the current quasi-standard, JPEG2000. However, the reasons for this success are not yet fully understood, and PDE-based compression is still in a proof-of-concept stage. With a probabilistic justification for anisotropic diffusion, we contribute to a deeper insight into design principles for PDE-based codecs. Moreover, by analysing the interaction between efficient storage methods and image reconstruction with diffusion, we can rank PDEs according to their practical value in compression. Based on these observations, we advance PDE-based compression towards practical viability: First, we present a new hybrid codec that combines PDE- and patch-based interpolation to deal with highly textured images. Furthermore, a new video player demonstrates the real-time capacities of PDE-based image interpolation and a new region of interest coding algorithm represents important image areas with high accuracy. Finally, we propose a new framework for diffusion-based image colourisation that we use to build an efficient codec for colour images. Experiments on real world image databases show that our new method is qualitatively competitive to current state-of-the-art codecs.Diese Dissertation ist der Bildkompression mit partiellen Differentialgleichungen (PDEs, partial differential equations) gewidmet. PDE-Codecs speichern nur einen geringen Anteil aller Bildpunkte und transportieren deren Information in fehlende Bildregionen. In einigen Fällen kann PDE-basierte Kompression den aktuellen Quasi-Standard, JPEG2000, bereits schlagen. Allerdings sind die Gründe für diesen Erfolg noch nicht vollständig erforscht, und PDE-basierte Kompression befindet sich derzeit noch im Anfangsstadium. Wir tragen durch eine probabilistische Rechtfertigung anisotroper Diffusion zu einem tieferen Verständnis PDE-basierten Codec-Designs bei. Eine Analyse der Interaktion zwischen effizienten Speicherverfahren und Bildrekonstruktion erlaubt es uns, PDEs nach ihrem Nutzen für die Kompression zu beurteilen. Anhand dieser Einsichten entwickeln wir PDE-basierte Kompression hinsichtlich ihrer praktischen Nutzbarkeit weiter: Wir stellen einen Hybrid-Codec für hochtexturierte Bilder vor, der umgebungsbasierte Interpolation mit PDEs kombiniert. Ein neuer Video-Dekodierer demonstriert die Echtzeitfähigkeit PDE-basierter Interpolation und eine Region-of-Interest-Methode erlaubt es, wichtige Bildbereiche mit hoher Genauigkeit zu speichern. Schlussendlich stellen wir ein neues diffusionsbasiertes Kolorierungsverfahren vor, welches uns effiziente Kompression von Farbbildern ermöglicht. Experimente auf Realwelt-Bilddatenbanken zeigen die Konkurrenzfähigkeit dieses Verfahrens auf