GMRES-Accelerated ADMM for Quadratic Objectives

We consider the sequence acceleration problem for the alternating direction method-of-multipliers (ADMM) applied to a class of equality-constrained problems with strongly convex quadratic objectives, which frequently arise as the Newton subproblem of interior-point methods. Within this context, the ADMM update equations are linear, the iterates are confined within a Krylov subspace, and the General Minimum RESidual (GMRES) algorithm is optimal in its ability to accelerate convergence. The basic ADMM method solves a $\kappa$-conditioned problem in $O(\sqrt{\kappa})$ iterations. We give theoretical justification and numerical evidence that the GMRES-accelerated variant consistently solves the same problem in $O(\kappa^{1/4})$ iterations for an order-of-magnitude reduction in iterations, despite a worst-case bound of $O(\sqrt{\kappa})$ iterations. The method is shown to be competitive against standard preconditioned Krylov subspace methods for saddle-point problems. The method is embedded within SeDuMi, a popular open-source solver for conic optimization written in MATLAB, and used to solve many large-scale semidefinite programs with error that decreases like $O(1/k^{2})$, instead of $O(1/k)$, where $k$ is the iteration index.Comment: 31 pages, 7 figures. Accepted for publication in SIAM Journal on Optimization (SIOPT

Generalized Forward-Backward Splitting

This paper introduces the generalized forward-backward splitting algorithm for minimizing convex functions of the form $F + \sum_{i=1}^n G_i$, where $F$ has a Lipschitz-continuous gradient and the $G_i$'s are simple in the sense that their Moreau proximity operators are easy to compute. While the forward-backward algorithm cannot deal with more than $n = 1$ non-smooth function, our method generalizes it to the case of arbitrary $n$. Our method makes an explicit use of the regularity of $F$ in the forward step, and the proximity operators of the $G_i$'s are applied in parallel in the backward step. This allows the generalized forward backward to efficiently address an important class of convex problems. We prove its convergence in infinite dimension, and its robustness to errors on the computation of the proximity operators and of the gradient of $F$. Examples on inverse problems in imaging demonstrate the advantage of the proposed methods in comparison to other splitting algorithms.Comment: 24 pages, 4 figure

Iterative splitting schemes for a soft material poromechanics model

We address numerical solvers for a poromechanics model particularly adapted for soft materials, as it generally respects thermodynamics principles and energy balance. Considering the multi-physics nature of the problem, which involves solid and fluid species, interacting on the basis of mass balance and momentum conservation, we decide to adopt a solution strategy of the discrete problem based on iterative splitting schemes. As the model is similar (but not equivalent to) the Biot poromechanics problem, we follow the abundant literature for solvers of the latter equations, developing two approaches that resemble the well known undrained and fixed-stress splits for the Biot model. A thorough convergence analysis of the proposed schemes is performed. In particular, the undrained-like split is developed and analyzed in the framework of generalized gradient flows, whereas the fixed-stress-like split is understood as block-diagonal -type stabilization and analyzed by means of a relative stability analysis. In addition, the application of Anderson acceleration is suggested, improving the robustness of the split schemes. Finally, we test these methods on different benchmark tests, and we also compare their performance with respect to a monolithic approach. Together with the theoretical analysis, the numerical examples provide guidelines to appropriately choose what split scheme shall be used to address realistic applications of the soft material poromechanics model.publishedVersio

Blind Ptychographic Phase Retrieval via Convergent Alternating Direction Method of Multipliers

Ptychography has risen as a reference X-ray imaging technique: it achieves resolutions of one billionth of a meter, macroscopic field of view, or the capability to retrieve chemical or magnetic contrast, among other features. A ptychographyic reconstruction is normally formulated as a blind phase retrieval problem, where both the image (sample) and the probe (illumination) have to be recovered from phaseless measured data. In this article we address a nonlinear least squares model for the blind ptychography problem with constraints on the image and the probe by maximum likelihood estimation of the Poisson noise model. We formulate a variant model that incorporates the information of phaseless measurements of the probe to eliminate possible artifacts. Next, we propose a generalized alternating direction method of multipliers designed for the proposed nonconvex models with convergence guarantee under mild conditions, where their subproblems can be solved by fast element-wise operations. Numerically, the proposed algorithm outperforms state-of-the-art algorithms in both speed and image quality.Comment: 23 page

First order algorithms in variational image processing

Variational methods in imaging are nowadays developing towards a quite universal and flexible tool, allowing for highly successful approaches on tasks like denoising, deblurring, inpainting, segmentation, super-resolution, disparity, and optical flow estimation. The overall structure of such approaches is of the form ${\cal D}(Ku) + \alpha {\cal R} (u) \rightarrow \min_u$ ; where the functional ${\cal D}$ is a data fidelity term also depending on some input data $f$ and measuring the deviation of $Ku$ from such and ${\cal R}$ is a regularization functional. Moreover $K$ is a (often linear) forward operator modeling the dependence of data on an underlying image, and $\alpha$ is a positive regularization parameter. While ${\cal D}$ is often smooth and (strictly) convex, the current practice almost exclusively uses nonsmooth regularization functionals. The majority of successful techniques is using nonsmooth and convex functionals like the total variation and generalizations thereof or $\ell_1$-norms of coefficients arising from scalar products with some frame system. The efficient solution of such variational problems in imaging demands for appropriate algorithms. Taking into account the specific structure as a sum of two very different terms to be minimized, splitting algorithms are a quite canonical choice. Consequently this field has revived the interest in techniques like operator splittings or augmented Lagrangians. Here we shall provide an overview of methods currently developed and recent results as well as some computational studies providing a comparison of different methods and also illustrating their success in applications.Comment: 60 pages, 33 figure