VDIP-TGV: Blind Image Deconvolution via Variational Deep Image Prior Empowered by Total Generalized Variation

Recovering clear images from blurry ones with an unknown blur kernel is a challenging problem. Deep image prior (DIP) proposes to use the deep network as a regularizer for a single image rather than as a supervised model, which achieves encouraging results in the nonblind deblurring problem. However, since the relationship between images and the network architectures is unclear, it is hard to find a suitable architecture to provide sufficient constraints on the estimated blur kernels and clean images. Also, DIP uses the sparse maximum a posteriori (MAP), which is insufficient to enforce the selection of the recovery image. Recently, variational deep image prior (VDIP) was proposed to impose constraints on both blur kernels and recovery images and take the standard deviation of the image into account during the optimization process by the variational principle. However, we empirically find that VDIP struggles with processing image details and tends to generate suboptimal results when the blur kernel is large. Therefore, we combine total generalized variational (TGV) regularization with VDIP in this paper to overcome these shortcomings of VDIP. TGV is a flexible regularization that utilizes the characteristics of partial derivatives of varying orders to regularize images at different scales, reducing oil painting artifacts while maintaining sharp edges. The proposed VDIP-TGV effectively recovers image edges and details by supplementing extra gradient information through TGV. Additionally, this model is solved by the alternating direction method of multipliers (ADMM), which effectively combines traditional algorithms and deep learning methods. Experiments show that our proposed VDIP-TGV surpasses various state-of-the-art models quantitatively and qualitatively.Comment: 13 pages, 5 figure

Bayesian computation in imaging inverse problems with partially unknown models

Many imaging problems require solving a high-dimensional inverse problem that is ill-conditioned or ill-posed. Imaging methods typically address this difficulty by regularising the estimation problem to make it well-posed. This often requires setting the value of the so-called regularisation parameters that control the amount of regularisation enforced. These parameters are notoriously difficult to set a priori and can have a dramatic impact on the recovered estimates. In this thesis, we propose a general empirical Bayesian method for setting regularisation parameters in imaging problems that are convex w.r.t. the unknown image. Our method calibrates regularisation parameters directly from the observed data by maximum marginal likelihood estimation, and can simultaneously estimate multiple regularisation parameters. A main novelty is that this maximum marginal likelihood estimation problem is efficiently solved by using a stochastic proximal gradient algorithm that is driven by two proximal Markov chain Monte Carlo samplers, thus intimately combining modern high-dimensional optimisation and stochastic sampling techniques. Furthermore, the proposed algorithm uses the same basic operators as proximal optimisation algorithms, namely gradient and proximal operators, and it is therefore straightforward to apply to problems that are currently solved by using proximal optimisation techniques. We also present a detailed theoretical analysis of the proposed methodology, and demonstrate it with a range of experiments and comparisons with alternative approaches from the literature. The considered experiments include image denoising, non-blind image deconvolution, and hyperspectral unmixing, using synthesis and analysis priors involving the `1, total-variation, total-variation and `1, and total-generalised-variation pseudo-norms. Moreover, we explore some other applications of the proposed method including maximum marginal likelihood estimation in Bayesian logistic regression and audio compressed sensing, as well as an application to model selection based on residuals

Restaurierung von kohärenten Bildern

In this thesis a series of novel algorithms for high quality restoration of coherent images is introduced. This task cannot be solved with established methods for the restoration of incoherent images. These algorithms focus on the correction of images in coherent imaging systems with a-priori known aberrations. The new wavefront correction algorithms achieve a significantly higher restoration quality than any previously known technique. The algorithms in this thesis are based on latest advances in optimization algorithms, particularly projections onto convex sets, proximal optimization and fractal self-similarity. Convergence and performance of the individual algorithms are analyzed in detail in various scenarios on real and simulated images. The evaluation also deals with the impact of noise on the restoration quality. Practical application of the new algorithms on microscopic images of diverse biological and human samples, as well as shadowgraph images of plankton acquired with a laboratory setup prove their efficiency. The new algorithms also have promising future applications in other areas, for example in adaptive optics and astronomy.In dieser Thesis werden mehrere neue Algorithmen für eine qualitativ hochwertige Restaurierung von kohärenten Bildern vorgestellt. Diese Aufgabe kann mit den bekannten Methoden für die Restaurierung von nicht kohärenten Bildern nicht gelöst werden. Die neuen Algorithmen sind auf die Wiederherstellung von Bildern in kohärenten Abbildungssystemen, bei denen die Aberrationen a-priori bekannt sind, ausgerichtet. Sie dienen der Korrektur der Wellenfront und erreichen eine wesentlich höhere Qualität der Bildrekonstruktion als sämtliche vorbekannte Verfahren. Die Algorithmen in dieser Thesis basieren auf neuesten Optimierungsalgorithmen, wie Projektionen in konvexe Sets, proximale Optimierung und fraktaler Ähnlichkeit. Die Konvergenz und Leistung der einzelnen Algorithmen wird ausführlich in unterschiedlichen Szenarien mit simulierten und realen Bildern untersucht. Eine praktische Erprobung der neuen Algorithmen an mikroskopischen Aufnahmen von unterschiedlichen biologischen und humanen Proben, wie auch an Aufnahmen vom Shadowgraph, bestätigt ihre Effizienz. Die neuen Algorithmen haben vielversprechende künftige Anwendungen, auch in anderen Gebieten, z.B. in der adaptiven Optik und der Astronomie

Computational Inverse Problems

Inverse problem typically deal with the identification of unknown quantities from indirect measurements and appear in many areas in technology, medicine, biology, finance, and econometrics. The computational solution of such problems is a very active, interdisciplinary field with close connections to optimization, control theory, differential equations, asymptotic analysis, statistics, and probability. The focus of this workshop was on hybrid methods, model reduction, regularization in Banach spaces, and statistical approaches

Learning to process with spikes and to localise pulses

In the last few decades, deep learning with artificial neural networks (ANNs) has emerged as one of the most widely used techniques in tasks such as classification and regression, achieving competitive results and in some cases even surpassing human-level performance. Nonetheless, as ANN architectures are optimised towards empirical results and departed from their biological precursors, how exactly human brains process information using these short electrical pulses called spikes remains a mystery. Hence, in this thesis, we explore the problem of learning to process with spikes and to localise pulses. We first consider spiking neural networks (SNNs), a type of ANN that more closely mimic biological neural networks in that neurons communicate with one another using spikes. This unique architecture allows us to look into the role of heterogeneity in learning. Since it is conjectured that the information is encoded by the timing of spikes, we are particularly interested in the heterogeneity of time constants of neurons. We then trained SNNs for classification tasks on a range of visual and auditory neuromorphic datasets, which contain streams of events (spike times) instead of the conventional frame-based data, and show that the overall performance is improved by allowing the neurons to have different time constants, especially on tasks with richer temporal structure. We also find that the learned time constants are distributed similarly to those experimentally observed in some mammalian cells. Besides, we demonstrate that learning with heterogeneity improves robustness against hyperparameter mistuning. These results suggest that heterogeneity may be more than the byproduct of noisy processes and perhaps serves a key role in learning in changing environments, yet heterogeneity has been overlooked in basic artificial models. While neuromorphic datasets, which are often captured by neuromorphic devices that closely model the corresponding biological systems, have enabled us to explore the more biologically plausible SNNs, there still exists a gap in understanding how spike times encode information in actual biological neural networks like human brains, as such data is difficult to acquire due to the trade-off between the timing precision and the number of cells simultaneously recorded electrically. Instead, what we usually obtain is the low-rate discrete samples of trains of filtered spikes. Hence, in the second part of the thesis, we focus on a different type of problem involving pulses, that is to retrieve the precise pulse locations from these low-rate samples. We make use of the finite rate of innovation (FRI) sampling theory, which states that perfect reconstruction is possible for classes of continuous non-bandlimited signals that have a small number of free parameters. However, existing FRI methods break down under very noisy conditions due to the so-called subspace swap event. Thus, we present two novel model-based learning architectures: Deep Unfolded Projected Wirtinger Gradient Descent (Deep Unfolded PWGD) and FRI Encoder-Decoder Network (FRIED-Net). The former is based on the existing iterative denoising algorithm for subspace-based methods, while the latter models directly the relationship between the samples and the locations of the pulses using an autoencoder-like network. Using a stream of K Diracs as an example, we show that both algorithms are able to overcome the breakdown inherent in the existing subspace-based methods. Moreover, we extend our FRIED-Net framework beyond conventional FRI methods by considering when the shape is unknown. We show that the pulse shape can be learned using backpropagation. This coincides with the application of spike detection from real-world calcium imaging data, where we achieve competitive results. Finally, we explore beyond canonical FRI signals and demonstrate that FRIED-Net is able to reconstruct streams of pulses with different shapes.Open Acces

Computational Imaging Approach to Recovery of Target Coordinates Using Orbital Sensor Data

This dissertation addresses the components necessary for simulation of an image-based recovery of the position of a target using orbital image sensors. Each component is considered in detail, focusing on the effect that design choices and system parameters have on the accuracy of the position estimate. Changes in sensor resolution, varying amounts of blur, differences in image noise level, selection of algorithms used for each component, and lag introduced by excessive processing time all contribute to the accuracy of the result regarding recovery of target coordinates using orbital sensor data. Using physical targets and sensors in this scenario would be cost-prohibitive in the exploratory setting posed, therefore a simulated target path is generated using Bezier curves which approximate representative paths followed by the targets of interest. Orbital trajectories for the sensors are designed on an elliptical model representative of the motion of physical orbital sensors. Images from each sensor are simulated based on the position and orientation of the sensor, the position of the target, and the imaging parameters selected for the experiment (resolution, noise level, blur level, etc.). Post-processing of the simulated imagery seeks to reduce noise and blur and increase resolution. The only information available for calculating the target position by a fully implemented system are the sensor position and orientation vectors and the images from each sensor. From these data we develop a reliable method of recovering the target position and analyze the impact on near-realtime processing. We also discuss the influence of adjustments to system components on overall capabilities and address the potential system size, weight, and power requirements from realistic implementation approaches

Advanced Bayesian Monte Carlo Methods for Inference and Control

Monte Carlo methods are are an ubiquitous tool in modern statistics. Under the Bayesian paradigm, they are used for estimating otherwise intractable integrals arising when integrating a function $h$ with respect to a posterior distribution $\pi$. This thesis discusses several aspects of such Monte Carlo methods. The first discussion evolves around the problem of sampling from only almost everywhere differentiable distributions, a class of distributions which includes all log-concave posteriors. A new sampling method based on a second-order diffusion process is proposed, new theoretical results are proved, and extensive numerical illustrations elucidate the benefits and weaknesses of various methods applicable in these settings. In high-dimensional settings, one can exploit local structures of inverse problems to parallelise computations. This will be explored in both fully localisable problems, and problems where conditional independence of variables given some others holds only approximately. This thesis proposes two algorithms using parallelisation techniques, and shows their empirical performance on two localisable imaging problems. Another problem arises when defining function space priors over high-dimensional domains. The commonly used Karhunen-Loève priors suffer from bad dimensional scaling: they require an orthogonal basis of the function space, which can often be obtained as a product of one-dimensional basis functions. This leads to the number of parameters growing exponentially in the dimension $d$ of the function domain. The trace-class neural network prior, proposed in this thesis, scales more favourably in the dimension of the function's domain. This prior is a Bayesian neural network prior, where each weight and bias has an independent Gaussian prior, but with a key difference to existing Bayesian neural network priors: the variances decrease in the width of the network, such that the variances form a summable sequence and the infinite width limit neural network is well defined. As is shown in this thesis, the resulting posterior of the unknown function is amenable to sampling using Hilbert space Markov chain Monte Carlo methods. These sampling methods are favoured because they are stable under mesh-refinement, in the sense that the acceptance probability does not shrink to 0 as more parameters are introduced to better approximate the well-defined infinite limit. Both numerical illustrations and theoretical results show that these priors are competitive and have distinct advantages over other function space priors. These different function space priors are then used in stochastic control. To this end, a suitable likelihood for continuous value functions in a Bayesian approach to reinforcement learning is defined. This thesis proves that it can be used in conjunction with both the classical Karhunen-Loève prior and the proposed trace-class neural network prior. Numerical examples compare the resulting posteriors, and illustrate the new prior's performance and dimension robustness.Cantab Capital Institute for the Mathematics of Informatio