86 research outputs found
Effective Modified Hybrid Conjugate Gradient Method for Large-Scale Symmetric Nonlinear Equations
In this paper, we proposed hybrid conjugate gradient method using the convex combination of FR and PRP conjugate gradient methods for solving Large-scale symmetric nonlinear equations via Andrei approach with nonmonotone line search. Logical formula for obtaining the convex parameter using Newton and our proposed directions was also proposed. Under appropriate conditions global convergence was established. Reported numerical results show that the proposed method is very promising
Some Unconstrained Optimization Methods
Although it is a very old theme, unconstrained optimization is an area which is always actual for many scientists. Today, the results of unconstrained optimization are applied in different branches of science, as well as generally in practice. Here, we present the line search techniques. Further, in this chapter we consider some unconstrained optimization methods. We try to present these methods but also to present some contemporary results in this area
A New Hybrid Approach for Solving Large-scale Monotone Nonlinear Equations
In this paper, a new hybrid conjugate gradient method for solving monotone nonlinear equations is introduced. The scheme is a combination of the Fletcher-Reeves (FR) and Polak-Ribiére-Polyak (PRP) conjugate gradient methods with the Solodov and Svaiter projection strategy. Using suitable assumptions, the global convergence of the scheme with monotone line search is provided. Lastly, a numerical experiment was used to enumerate the suitability of the proposed scheme for large-scale problems
Regularization of Limited Memory Quasi-Newton Methods for Large-Scale Nonconvex Minimization
This paper deals with regularized Newton methods, a flexible class of
unconstrained optimization algorithms that is competitive with line search and
trust region methods and potentially combines attractive elements of both. The
particular focus is on combining regularization with limited memory
quasi-Newton methods by exploiting the special structure of limited memory
algorithms. Global convergence of regularization methods is shown under mild
assumptions and the details of regularized limited memory quasi-Newton updates
are discussed including their compact representations.
Numerical results using all large-scale test problems from the CUTEst
collection indicate that our regularized version of L-BFGS is competitive with
state-of-the-art line search and trust-region L-BFGS algorithms and previous
attempts at combining L-BFGS with regularization, while potentially
outperforming some of them, especially when nonmonotonicity is involved.Comment: 23 pages, 4 figure
Nonmonotone Barzilai-Borwein Gradient Algorithm for -Regularized Nonsmooth Minimization in Compressive Sensing
This paper is devoted to minimizing the sum of a smooth function and a
nonsmooth -regularized term. This problem as a special cases includes
the -regularized convex minimization problem in signal processing,
compressive sensing, machine learning, data mining, etc. However, the
non-differentiability of the -norm causes more challenging especially
in large problems encountered in many practical applications. This paper
proposes, analyzes, and tests a Barzilai-Borwein gradient algorithm. At each
iteration, the generated search direction enjoys descent property and can be
easily derived by minimizing a local approximal quadratic model and
simultaneously taking the favorable structure of the -norm. Moreover, a
nonmonotone line search technique is incorporated to find a suitable stepsize
along this direction. The algorithm is easily performed, where the values of
the objective function and the gradient of the smooth term are required at
per-iteration. Under some conditions, the proposed algorithm is shown to be
globally convergent. The limited experiments by using some nonconvex
unconstrained problems from CUTEr library with additive -regularization
illustrate that the proposed algorithm performs quite well. Extensive
experiments for -regularized least squares problems in compressive
sensing verify that our algorithm compares favorably with several
state-of-the-art algorithms which are specifically designed in recent years.Comment: 20 page
On Quasi-Newton Forward--Backward Splitting: Proximal Calculus and Convergence
We introduce a framework for quasi-Newton forward--backward splitting
algorithms (proximal quasi-Newton methods) with a metric induced by diagonal
rank- symmetric positive definite matrices. This special type of
metric allows for a highly efficient evaluation of the proximal mapping. The
key to this efficiency is a general proximal calculus in the new metric. By
using duality, formulas are derived that relate the proximal mapping in a
rank- modified metric to the original metric. We also describe efficient
implementations of the proximity calculation for a large class of functions;
the implementations exploit the piece-wise linear nature of the dual problem.
Then, we apply these results to acceleration of composite convex minimization
problems, which leads to elegant quasi-Newton methods for which we prove
convergence. The algorithm is tested on several numerical examples and compared
to a comprehensive list of alternatives in the literature. Our quasi-Newton
splitting algorithm with the prescribed metric compares favorably against
state-of-the-art. The algorithm has extensive applications including signal
processing, sparse recovery, machine learning and classification to name a few.Comment: arXiv admin note: text overlap with arXiv:1206.115
A Simple Sufficient Descent Method for Unconstrained Optimization
We develop a sufficient descent method for solving large-scale unconstrained optimization problems. At each iteration, the search direction is a linear combination of the gradient
at the current and the previous steps. An attractive property of this method is that the generated directions are always descent. Under some appropriate conditions, we show that the proposed
method converges globally. Numerical experiments on some unconstrained minimization problems
from CUTEr library are reported, which illustrate that the proposed method is promising
- …