Convex regularization of discrete-valued inverse problems

This work is concerned with linear inverse problems where a distributed parameter is known a priori to only take on values from a given discrete set. This property can be promoted in Tikhonov regularization with the aid of a suitable convex but nondifferentiable regularization term. This allows applying standard approaches to show well-posedness and convergence rates in Bregman distance. Using the specific properties of the regularization term, it can be shown that convergence (albeit without rates) actually holds pointwise. Furthermore, the resulting Tikhonov functional can be minimized efficiently using a semi-smooth Newton method. Numerical examples illustrate the properties of the regularization term and the numerical solution

Conditions on optimal support recovery in unmixing problems by means of multi-penalty regularization

Inspired by several real-life applications in audio processing and medical image analysis, where the quantity of interest is generated by several sources to be accurately modeled and separated, as well as by recent advances in regularization theory and optimization, we study the conditions on optimal support recovery in inverse problems of unmixing type by means of multi-penalty regularization. We consider and analyze a regularization functional composed of a data-fidelity term, where signal and noise are additively mixed, a non-smooth, convex, sparsity promoting term, and a quadratic penalty term to model the noise. We prove not only that the well-established theory for sparse recovery in the single parameter case can be translated to the multi-penalty settings, but we also demonstrate the enhanced properties of multi-penalty regularization in terms of support identification compared to sole $\ell^1$-minimization. We additionally confirm and support the theoretical results by extensive numerical simulations, which give a statistics of robustness of the multi-penalty regularization scheme with respect to the single-parameter counterpart. Eventually, we confirm a significant improvement in performance compared to standard $\ell^1$-regularization for compressive sensing problems considered in our experiments

A Regularization Term Based on a Discrete Total Variation for Mathematical Image Processing

In this paper, a new regularization term is proposed to solve mathematical image problems. By using difference operators in the four directions; horizontal, vertical and two diagonal directions, an estimation of derivative amplitude is found. Based on the new obtained estimation, a new regularization term will be defined, which can be viewed as a new discretized total variation (TVprn) model. By improving TVprn, a more effective regularization term is introduced. By finding conjugate of TVprn and producing vector fields with special constraints, a new discretized TV for two dimensional discrete functions is proposed (TVnew). The capability of the new TV model to solve mathematical image problems is examined in some numerical experiments. It is shown that the new proposed TV model can reconstruct the edges and corners of the noisy images better than other TVs. Moreover, two test experiments of resolution enhancement problem are solved and compared with some other different TVs

3D eclipse mapping in AM Herculis systems - ‘genetically modified fireflies’

In order to map the three-dimensional location and shape of the emission originating within the accretion stream in AM Her systems, we have investigated the possibilities of relaxing the hitherto-applied constraint of a predetermined stream trajectory in modelling the eclipse profiles. We use emission points which can be located anywhere in the Roche lobe of the primary, together with a regularization term which favours any curved stream structure, connected at the secondary and white dwarf primary. Our results show that, given suitable regularization constraints, such inversion is feasible. We investigate the effect of removing the regularization term, and also the sensitivity of the fit to input parameters such as inclination

Takeuchi's Information Criteria as a form of Regularization

Takeuchi's Information Criteria (TIC) is a linearization of maximum likelihood estimator bias which shrinks the model parameters towards the maximum entropy distribution, even when the model is mis-specified. In statistical machine learning, $L_2$ regularization (a.k.a. ridge regression) also introduces a parameterized bias term with the goal of minimizing out-of-sample entropy, but generally requires a numerical solver to find the regularization parameter. This paper presents a novel regularization approach based on TIC; the approach does not assume a data generation process and results in a higher entropy distribution through more efficient sample noise suppression. The resulting objective function can be directly minimized to estimate and select the best model, without the need to select a regularization parameter, as in ridge regression. Numerical results applied to a synthetic high dimensional dataset generated from a logistic regression model demonstrate superior model performance when using the TIC based regularization over a $L_1$ and a $L_2$ penalty term

Iterative CT reconstruction using shearlet-based regularization

In computerized tomography, it is important to reduce the image noise without increasing the acquisition dose. Extensive research has been done into total variation minimization for image denoising and sparse-view reconstruction. However, TV minimization methods show superior denoising performance for simple images (with little texture), but result in texture information loss when applied to more complex images. Since in medical imaging, we are often confronted with textured images, it might not be beneficial to use TV. Our objective is to find a regularization term outperforming TV for sparse-view reconstruction and image denoising in general. A recent efficient solver was developed for convex problems, based on a split-Bregman approach, able to incorporate regularization terms different from TV. In this work, a proof-of-concept study demonstrates the usage of the discrete shearlet transform as a sparsifying transform within this solver for CT reconstructions. In particular, the regularization term is the 1-norm of the shearlet coefficients. We compared our newly developed shearlet approach to traditional TV on both sparse-view and on low-count simulated and measured preclinical data. Shearlet-based regularization does not outperform TV-based regularization for all datasets. Reconstructed images exhibit small aliasing artifacts in sparse-view reconstruction problems, but show no staircasing effect. This results in a slightly higher resolution than with TV-based regularization

Convergence Rates for Inverse Problems with Impulsive Noise

We study inverse problems F(f) = g with perturbed right hand side g^{obs} corrupted by so-called impulsive noise, i.e. noise which is concentrated on a small subset of the domain of definition of g. It is well known that Tikhonov-type regularization with an L^1 data fidelity term yields significantly more accurate results than Tikhonov regularization with classical L^2 data fidelity terms for this type of noise. The purpose of this paper is to provide a convergence analysis explaining this remarkable difference in accuracy. Our error estimates significantly improve previous error estimates for Tikhonov regularization with L^1-fidelity term in the case of impulsive noise. We present numerical results which are in good agreement with the predictions of our analysis

Minimization of multi-penalty functionals by alternating iterative thresholding and optimal parameter choices

Inspired by several recent developments in regularization theory, optimization, and signal processing, we present and analyze a numerical approach to multi-penalty regularization in spaces of sparsely represented functions. The sparsity prior is motivated by the largely expected geometrical/structured features of high-dimensional data, which may not be well-represented in the framework of typically more isotropic Hilbert spaces. In this paper, we are particularly interested in regularizers which are able to correctly model and separate the multiple components of additively mixed signals. This situation is rather common as pure signals may be corrupted by additive noise. To this end, we consider a regularization functional composed by a data-fidelity term, where signal and noise are additively mixed, a non-smooth and non-convex sparsity promoting term, and a penalty term to model the noise. We propose and analyze the convergence of an iterative alternating algorithm based on simple iterative thresholding steps to perform the minimization of the functional. By means of this algorithm, we explore the effect of choosing different regularization parameters and penalization norms in terms of the quality of recovering the pure signal and separating it from additive noise. For a given fixed noise level numerical experiments confirm a significant improvement in performance compared to standard one-parameter regularization methods. By using high-dimensional data analysis methods such as Principal Component Analysis, we are able to show the correct geometrical clustering of regularized solutions around the expected solution. Eventually, for the compressive sensing problems considered in our experiments we provide a guideline for a choice of regularization norms and parameters.Comment: 32 page

The representer theorem for Hilbert spaces: a necessary and sufficient condition

A family of regularization functionals is said to admit a linear representer theorem if every member of the family admits minimizers that lie in a fixed finite dimensional subspace. A recent characterization states that a general class of regularization functionals with differentiable regularizer admits a linear representer theorem if and only if the regularization term is a non-decreasing function of the norm. In this report, we improve over such result by replacing the differentiability assumption with lower semi-continuity and deriving a proof that is independent of the dimensionality of the space