311 research outputs found
Nonmonotone spectral projected gradient methods on convex sets
Nonmonotone projected gradient techniques are considered for the minimization of differentiable functions on closed convex sets. The classical projected gradient schemes are extended to include a nonmonotone steplength strategy that is based on the Grippo-Lampariello-Lucidi nonmonotone line search. In particular, the nonmonotone strategy is combined with the spectral gradient choice of steplength to accelerate the convergence process. In addition to the classical projected gradient nonlinear path, the feasible spectral projected gradient is used as a search direction to avoid additional trial projections during the one-dimensional search process. Convergence properties and extensive numerical results are presented.1041196121
Convergence properties of nonmonotone spectral projected gradient methods
AbstractIn a recent paper, a nonmonotone spectral projected gradient (SPG) method was introduced by Birgin et al. for the minimization of differentiable functions on closed convex sets and extensive presented results showed that this method was very efficient. In this paper, we give a more comprehensive theoretical analysis of the SPG method. In doing so, we remove various boundedness conditions that are assumed in existing results, such as boundedness from below of f, boundedness of xk or existence of accumulation point of {xk}. If ∇f(·) is uniformly continuous, we establish the convergence theory of this method and prove that the SPG method forces the sequence of projected gradients to zero. Moreover, we show under appropriate conditions that the SPG method has some encouraging convergence properties, such as the global convergence of the sequence of iterates generated by this method and the finite termination, etc. Therefore, these results show that the SPG method is attractive in theory
An optimal subgradient algorithm for large-scale convex optimization in simple domains
This paper shows that the optimal subgradient algorithm, OSGA, proposed in
\cite{NeuO} can be used for solving structured large-scale convex constrained
optimization problems. Only first-order information is required, and the
optimal complexity bounds for both smooth and nonsmooth problems are attained.
More specifically, we consider two classes of problems: (i) a convex objective
with a simple closed convex domain, where the orthogonal projection on this
feasible domain is efficiently available; (ii) a convex objective with a simple
convex functional constraint. If we equip OSGA with an appropriate
prox-function, the OSGA subproblem can be solved either in a closed form or by
a simple iterative scheme, which is especially important for large-scale
problems. We report numerical results for some applications to show the
efficiency of the proposed scheme. A software package implementing OSGA for
above domains is available
An Augmented Lagrangian Approach for Sparse Principal Component Analysis
Principal component analysis (PCA) is a widely used technique for data
analysis and dimension reduction with numerous applications in science and
engineering. However, the standard PCA suffers from the fact that the principal
components (PCs) are usually linear combinations of all the original variables,
and it is thus often difficult to interpret the PCs. To alleviate this
drawback, various sparse PCA approaches were proposed in literature [15, 6, 17,
28, 8, 25, 18, 7, 16]. Despite success in achieving sparsity, some important
properties enjoyed by the standard PCA are lost in these methods such as
uncorrelation of PCs and orthogonality of loading vectors. Also, the total
explained variance that they attempt to maximize can be too optimistic. In this
paper we propose a new formulation for sparse PCA, aiming at finding sparse and
nearly uncorrelated PCs with orthogonal loading vectors while explaining as
much of the total variance as possible. We also develop a novel augmented
Lagrangian method for solving a class of nonsmooth constrained optimization
problems, which is well suited for our formulation of sparse PCA. We show that
it converges to a feasible point, and moreover under some regularity
assumptions, it converges to a stationary point. Additionally, we propose two
nonmonotone gradient methods for solving the augmented Lagrangian subproblems,
and establish their global and local convergence. Finally, we compare our
sparse PCA approach with several existing methods on synthetic, random, and
real data, respectively. The computational results demonstrate that the sparse
PCs produced by our approach substantially outperform those by other methods in
terms of total explained variance, correlation of PCs, and orthogonality of
loading vectors.Comment: 42 page
Adaptive First-Order Methods for General Sparse Inverse Covariance Selection
In this paper, we consider estimating sparse inverse covariance of a Gaussian
graphical model whose conditional independence is assumed to be partially
known. Similarly as in [5], we formulate it as an -norm penalized maximum
likelihood estimation problem. Further, we propose an algorithm framework, and
develop two first-order methods, that is, the adaptive spectral projected
gradient (ASPG) method and the adaptive Nesterov's smooth (ANS) method, for
solving this estimation problem. Finally, we compare the performance of these
two methods on a set of randomly generated instances. Our computational results
demonstrate that both methods are able to solve problems of size at least a
thousand and number of constraints of nearly a half million within a reasonable
amount of time, and the ASPG method generally outperforms the ANS method.Comment: 19 pages, 1 figur
- …