Elastic-Net Regularization: Error estimates and Active Set Methods

This paper investigates theoretical properties and efficient numerical algorithms for the so-called elastic-net regularization originating from statistics, which enforces simultaneously l^1 and l^2 regularization. The stability of the minimizer and its consistency are studied, and convergence rates for both a priori and a posteriori parameter choice rules are established. Two iterative numerical algorithms of active set type are proposed, and their convergence properties are discussed. Numerical results are presented to illustrate the features of the functional and algorithms

Beyond convergence rates: Exact recovery with Tikhonov regularization with sparsity constraints

The Tikhonov regularization of linear ill-posed problems with an $\ell^1$ penalty is considered. We recall results for linear convergence rates and results on exact recovery of the support. Moreover, we derive conditions for exact support recovery which are especially applicable in the case of ill-posed problems, where other conditions, e.g. based on the so-called coherence or the restricted isometry property are usually not applicable. The obtained results also show that the regularized solutions do not only converge in the $\ell^1$-norm but also in the vector space $\ell^0$ (when considered as the strict inductive limit of the spaces $\R^n$ as $n$ tends to infinity). Additionally, the relations between different conditions for exact support recovery and linear convergence rates are investigated. With an imaging example from digital holography the applicability of the obtained results is illustrated, i.e. that one may check a priori if the experimental setup guarantees exact recovery with Tikhonov regularization with sparsity constraints

Sparse Regularization with lql^q Penalty Term

We consider the stable approximation of sparse solutions to non-linear operator equations by means of Tikhonov regularization with a subquadratic penalty term. Imposing certain assumptions, which for a linear operator are equivalent to the standard range condition, we derive the usual convergence rate $O(\sqrt{\delta})$ of the regularized solutions in dependence of the noise level $\delta$. Particular emphasis lies on the case, where the true solution is known to have a sparse representation in a given basis. In this case, if the differential of the operator satisfies a certain injectivity condition, we can show that the actual convergence rate improves up to $O(\delta)$.Comment: 15 page

Optimal Convergence Rates for Tikhonov Regularization in Besov Scales

In this paper we deal with linear inverse problems and convergence rates for Tikhonov regularization. We consider regularization in a scale of Banach spaces, namely the scale of Besov spaces. We show that regularization in Banach scales differs from regularization in Hilbert scales in the sense that it is possible that stronger source conditions may lead to weaker convergence rates and vive versa. Moreover, we present optimal source conditions for regularization in Besov scales

Guaranteed optimal reachability control of reaction-diffusion equations using one-sided Lipschitz constants and model reduction

We show that, for any spatially discretized system of reaction-diffusion, the approximate solution given by the explicit Euler time-discretization scheme converges to the exact time-continuous solution, provided that diffusion coefficient be sufficiently large. By "sufficiently large", we mean that the diffusion coefficient value makes the one-sided Lipschitz constant of the reaction-diffusion system negative. We apply this result to solve a finite horizon control problem for a 1D reaction-diffusion example. We also explain how to perform model reduction in order to improve the efficiency of the method

Greedy Solution of Ill-Posed Problems: Error Bounds and Exact Inversion

The orthogonal matching pursuit (OMP) is an algorithm to solve sparse approximation problems. Sufficient conditions for exact recovery are known with and without noise. In this paper we investigate the applicability of the OMP for the solution of ill-posed inverse problems in general and in particular for two deconvolution examples from mass spectrometry and digital holography respectively. In sparse approximation problems one often has to deal with the problem of redundancy of a dictionary, i.e. the atoms are not linearly independent. However, one expects them to be approximatively orthogonal and this is quantified by the so-called incoherence. This idea cannot be transfered to ill-posed inverse problems since here the atoms are typically far from orthogonal: The ill-posedness of the operator causes that the correlation of two distinct atoms probably gets huge, i.e. that two atoms can look much alike. Therefore one needs conditions which take the structure of the problem into account and work without the concept of coherence. In this paper we develop results for exact recovery of the support of noisy signals. In the two examples in mass spectrometry and digital holography we show that our results lead to practically relevant estimates such that one may check a priori if the experimental setup guarantees exact deconvolution with OMP. Especially in the example from digital holography our analysis may be regarded as a first step to calculate the resolution power of droplet holography

Second order optimality conditions and their role in PDE control

If f : Rn R is twice continuously differentiable, f’(u) = 0 and f’’(u) is positive definite, then u is a local minimizer of f. This paper surveys the extension of this well known second order suffcient optimality condition to the case f : U R, where U is an infinite-dimensional linear normed space. The reader will be guided from the case of finite-dimensions via a brief discussion of the calculus of variations and the optimal control of ordinary differential equations to the control of nonlinear partial differential equations, where U is a function space. In particular, the following questions will be addressed: Is the extension to infinite dimensions straightforward or will unexpected difficulties occur? How second order sufficient optimality conditions must be modified, if simple inequality constraints are imposed on u? Why do we need second order conditions and how can they be applied? If they are important, are we able to check if they are fulfilled order sufficient optimality condition to the case f : U R, where U is an infinite-dimensional linear normed space. The reader will be guided from the case of finite-dimensions via a brief discussion of the calculus of variations and the optimal control of ordinary differential equations to the control of nonlinear partial differential equations, where U is a function space. In particular, the following questions will be addressed: Is the extension to infinite dimensions straightforward or will unexpected difficulties occur? How second order sufficient optimality conditions must be modified, if simple inequality constraints are imposed on u? Why do we need second order conditions and how can they be applied? If they are important, are we able to check if they are fulfilled? It turns out that infinite dimensions cause new difficulties that do not occur in finite dimensions. We will be faced with the surprising fact that the space, where f’’(u) exists can be useless to ensure positive definiteness of the quadratic form v f’’(u)v2. In this context, the famous two-norm discrepancy, its consequences, and techniques for overcoming this difficulty are explained. To keep the presentation simple, the theory is developed for problems in function spaces with simple box constraints of the form a = u = ß. The theory of second order conditions in the control of partial differential equations is presented exemplarily for the nonlinear heat equation. Different types of critical cones are introduced, where the positivity of f’’(u) must be required. Their form depends on whether a so-called Tikhonov regularization term is part of the functional f or not. In this context, the paper contains also new results that lead to quadratic growth conditions in the strong sense. As a first application of second-order sufficient conditions, the stability of optimal solutions with respect to perturbations of the data of the control problem is discussed. Second, their use in analyzing the discretization of control problems by finite elements is studied. A survey on further related topics, open questions, and relevant literature concludes the paper.The first author was partially supported by the Spanish Ministerio de Economía y Competitividad under project MTM2011-22711, the second author by DFG in the framework of the Collaborative Research Center SFB 910, project B6