146,186 research outputs found
Adaptive Preconditioned Gradient Descent with Energy
We propose an adaptive time step with energy for a large class of
preconditioned gradient descent methods, mainly applied to constrained
optimization problems. Our strategy relies on representing the usual descent
direction by the product of an energy variable and a transformed gradient, with
a preconditioning matrix, for example, to reflect the natural gradient induced
by the underlying metric in parameter space or to endow a projection operator
when linear equality constraints are present. We present theoretical results on
both unconditional stability and convergence rates for three respective classes
of objective functions. In addition, our numerical results shed light on the
excellent performance of the proposed method on several benchmark optimization
problems.Comment: 32 pages, 3 figure
Designing structured tight frames via an alternating projection method
Tight frames, also known as general Welch-bound- equality sequences, generalize orthonormal systems. Numerous applications - including communications, coding, and sparse approximation- require finite-dimensional tight frames that possess additional structural properties. This paper proposes an alternating projection method that is versatile enough to solve a huge class of inverse eigenvalue problems (IEPs), which includes the frame design problem. To apply this method, one needs only to solve a matrix nearness problem that arises naturally from the design specifications. Therefore, it is the fast and easy to develop versions of the algorithm that target new design problems. Alternating projection will often succeed even if algebraic constructions are unavailable. To demonstrate that alternating projection is an effective tool for frame design, the paper studies some important structural properties in detail. First, it addresses the most basic design problem: constructing tight frames with prescribed vector norms. Then, it discusses equiangular tight frames, which are natural dictionaries for sparse approximation. Finally, it examines tight frames whose individual vectors have low peak-to-average-power ratio (PAR), which is a valuable property for code-division multiple-access (CDMA) applications. Numerical experiments show that the proposed algorithm succeeds in each of these three cases. The appendices investigate the convergence properties of the algorithm
Projection methods in conic optimization
There exist efficient algorithms to project a point onto the intersection of
a convex cone and an affine subspace. Those conic projections are in turn the
work-horse of a range of algorithms in conic optimization, having a variety of
applications in science, finance and engineering. This chapter reviews some of
these algorithms, emphasizing the so-called regularization algorithms for
linear conic optimization, and applications in polynomial optimization. This is
a presentation of the material of several recent research articles; we aim here
at clarifying the ideas, presenting them in a general framework, and pointing
out important techniques
A Tensor-Based Dictionary Learning Approach to Tomographic Image Reconstruction
We consider tomographic reconstruction using priors in the form of a
dictionary learned from training images. The reconstruction has two stages:
first we construct a tensor dictionary prior from our training data, and then
we pose the reconstruction problem in terms of recovering the expansion
coefficients in that dictionary. Our approach differs from past approaches in
that a) we use a third-order tensor representation for our images and b) we
recast the reconstruction problem using the tensor formulation. The dictionary
learning problem is presented as a non-negative tensor factorization problem
with sparsity constraints. The reconstruction problem is formulated in a convex
optimization framework by looking for a solution with a sparse representation
in the tensor dictionary. Numerical results show that our tensor formulation
leads to very sparse representations of both the training images and the
reconstructions due to the ability of representing repeated features compactly
in the dictionary.Comment: 29 page
- …