235 research outputs found
A new steplength selection for scaled gradient methods with application to image deblurring
Gradient methods are frequently used in large scale image deblurring problems
since they avoid the onerous computation of the Hessian matrix of the objective
function. Second order information is typically sought by a clever choice of
the steplength parameter defining the descent direction, as in the case of the
well-known Barzilai and Borwein rules. In a recent paper, a strategy for the
steplength selection approximating the inverse of some eigenvalues of the
Hessian matrix has been proposed for gradient methods applied to unconstrained
minimization problems. In the quadratic case, this approach is based on a
Lanczos process applied every m iterations to the matrix of the most recent m
back gradients but the idea can be extended to a general objective function. In
this paper we extend this rule to the case of scaled gradient projection
methods applied to non-negatively constrained minimization problems, and we
test the effectiveness of the proposed strategy in image deblurring problems in
both the presence and the absence of an explicit edge-preserving regularization
term
An Inexact Successive Quadratic Approximation Method for Convex L-1 Regularized Optimization
We study a Newton-like method for the minimization of an objective function
that is the sum of a smooth convex function and an l-1 regularization term.
This method, which is sometimes referred to in the literature as a proximal
Newton method, computes a step by minimizing a piecewise quadratic model of the
objective function. In order to make this approach efficient in practice, it is
imperative to perform this inner minimization inexactly. In this paper, we give
inexactness conditions that guarantee global convergence and that can be used
to control the local rate of convergence of the iteration. Our inexactness
conditions are based on a semi-smooth function that represents a (continuous)
measure of the optimality conditions of the problem, and that embodies the
soft-thresholding iteration. We give careful consideration to the algorithm
employed for the inner minimization, and report numerical results on two test
sets originating in machine learning
A two-phase gradient method for quadratic programming problems with a single linear constraint and bounds on the variables
We propose a gradient-based method for quadratic programming problems with a
single linear constraint and bounds on the variables. Inspired by the GPCG
algorithm for bound-constrained convex quadratic programming [J.J. Mor\'e and
G. Toraldo, SIAM J. Optim. 1, 1991], our approach alternates between two phases
until convergence: an identification phase, which performs gradient projection
iterations until either a candidate active set is identified or no reasonable
progress is made, and an unconstrained minimization phase, which reduces the
objective function in a suitable space defined by the identification phase, by
applying either the conjugate gradient method or a recently proposed spectral
gradient method. However, the algorithm differs from GPCG not only because it
deals with a more general class of problems, but mainly for the way it stops
the minimization phase. This is based on a comparison between a measure of
optimality in the reduced space and a measure of bindingness of the variables
that are on the bounds, defined by extending the concept of proportioning,
which was proposed by some authors for box-constrained problems. If the
objective function is bounded, the algorithm converges to a stationary point
thanks to a suitable application of the gradient projection method in the
identification phase. For strictly convex problems, the algorithm converges to
the optimal solution in a finite number of steps even in case of degeneracy.
Extensive numerical experiments show the effectiveness of the proposed
approach.Comment: 30 pages, 17 figure
On the filtering effect of iterative regularization algorithms for linear least-squares problems
Many real-world applications are addressed through a linear least-squares
problem formulation, whose solution is calculated by means of an iterative
approach. A huge amount of studies has been carried out in the optimization
field to provide the fastest methods for the reconstruction of the solution,
involving choices of adaptive parameters and scaling matrices. However, in
presence of an ill-conditioned model and real data, the need of a regularized
solution instead of the least-squares one changed the point of view in favour
of iterative algorithms able to combine a fast execution with a stable
behaviour with respect to the restoration error. In this paper we want to
analyze some classical and recent gradient approaches for the linear
least-squares problem by looking at their way of filtering the singular values,
showing in particular the effects of scaling matrices and non-negative
constraints in recovering the correct filters of the solution
Global convergence enhancement of classical linesearch interior point methods for MCPs
AbstractRecent works have shown that a wide class of globally convergent interior point methods may manifest a weakness of convergence. Failures can be ascribed to the procedure of linesearch along the Newton step. In this paper, we introduce a globally convergent interior point method which performs backtracking along a piecewise linear path. Theoretical and computational results show the effectiveness of our proposal
- …