1,488 research outputs found
On Quasi-Newton Forward--Backward Splitting: Proximal Calculus and Convergence
We introduce a framework for quasi-Newton forward--backward splitting
algorithms (proximal quasi-Newton methods) with a metric induced by diagonal
rank- symmetric positive definite matrices. This special type of
metric allows for a highly efficient evaluation of the proximal mapping. The
key to this efficiency is a general proximal calculus in the new metric. By
using duality, formulas are derived that relate the proximal mapping in a
rank- modified metric to the original metric. We also describe efficient
implementations of the proximity calculation for a large class of functions;
the implementations exploit the piece-wise linear nature of the dual problem.
Then, we apply these results to acceleration of composite convex minimization
problems, which leads to elegant quasi-Newton methods for which we prove
convergence. The algorithm is tested on several numerical examples and compared
to a comprehensive list of alternatives in the literature. Our quasi-Newton
splitting algorithm with the prescribed metric compares favorably against
state-of-the-art. The algorithm has extensive applications including signal
processing, sparse recovery, machine learning and classification to name a few.Comment: arXiv admin note: text overlap with arXiv:1206.115
First order algorithms in variational image processing
Variational methods in imaging are nowadays developing towards a quite
universal and flexible tool, allowing for highly successful approaches on tasks
like denoising, deblurring, inpainting, segmentation, super-resolution,
disparity, and optical flow estimation. The overall structure of such
approaches is of the form ; where the functional is a data fidelity term also
depending on some input data and measuring the deviation of from such
and is a regularization functional. Moreover is a (often linear)
forward operator modeling the dependence of data on an underlying image, and
is a positive regularization parameter. While is often
smooth and (strictly) convex, the current practice almost exclusively uses
nonsmooth regularization functionals. The majority of successful techniques is
using nonsmooth and convex functionals like the total variation and
generalizations thereof or -norms of coefficients arising from scalar
products with some frame system. The efficient solution of such variational
problems in imaging demands for appropriate algorithms. Taking into account the
specific structure as a sum of two very different terms to be minimized,
splitting algorithms are a quite canonical choice. Consequently this field has
revived the interest in techniques like operator splittings or augmented
Lagrangians. Here we shall provide an overview of methods currently developed
and recent results as well as some computational studies providing a comparison
of different methods and also illustrating their success in applications.Comment: 60 pages, 33 figure
Projection methods in conic optimization
There exist efficient algorithms to project a point onto the intersection of
a convex cone and an affine subspace. Those conic projections are in turn the
work-horse of a range of algorithms in conic optimization, having a variety of
applications in science, finance and engineering. This chapter reviews some of
these algorithms, emphasizing the so-called regularization algorithms for
linear conic optimization, and applications in polynomial optimization. This is
a presentation of the material of several recent research articles; we aim here
at clarifying the ideas, presenting them in a general framework, and pointing
out important techniques
Real and Complex Monotone Communication Games
Noncooperative game-theoretic tools have been increasingly used to study many
important resource allocation problems in communications, networking, smart
grids, and portfolio optimization. In this paper, we consider a general class
of convex Nash Equilibrium Problems (NEPs), where each player aims to solve an
arbitrary smooth convex optimization problem. Differently from most of current
works, we do not assume any specific structure for the players' problems, and
we allow the optimization variables of the players to be matrices in the
complex domain. Our main contribution is the design of a novel class of
distributed (asynchronous) best-response- algorithms suitable for solving the
proposed NEPs, even in the presence of multiple solutions. The new methods,
whose convergence analysis is based on Variational Inequality (VI) techniques,
can select, among all the equilibria of a game, those that optimize a given
performance criterion, at the cost of limited signaling among the players. This
is a major departure from existing best-response algorithms, whose convergence
conditions imply the uniqueness of the NE. Some of our results hinge on the use
of VI problems directly in the complex domain; the study of these new kind of
VIs also represents a noteworthy innovative contribution. We then apply the
developed methods to solve some new generalizations of SISO and MIMO games in
cognitive radios and femtocell systems, showing a considerable performance
improvement over classical pure noncooperative schemes.Comment: to appear on IEEE Transactions in Information Theor
- âŠ