203 research outputs found
On complexity and convergence of high-order coordinate descent algorithms
Coordinate descent methods with high-order regularized models for
box-constrained minimization are introduced. High-order stationarity asymptotic
convergence and first-order stationarity worst-case evaluation complexity
bounds are established. The computer work that is necessary for obtaining
first-order -stationarity with respect to the variables of each
coordinate-descent block is whereas the computer
work for getting first-order -stationarity with respect to all the
variables simultaneously is . Numerical examples
involving multidimensional scaling problems are presented. The numerical
performance of the methods is enhanced by means of coordinate-descent
strategies for choosing initial points
Riemannian Sparse Coding for Positive Definite Matrices
International audienceInspired by the great success of sparse coding for vector valued data, our goal is to represent symmetric positive definite (SPD) data matrices as sparse linear combinations of atoms from a dictionary, where each atom itself is an SPD matrix. Since SPD matrices follow a non-Euclidean (in fact a Riemannian) geometry, existing sparse coding techniques for Euclidean data cannot be directly extended. Prior works have approached this problem by defining a sparse coding loss function using either extrinsic similarity measures (such as the log-Euclidean distance) or kernelized variants of statistical measures (such as the Stein divergence, Jeffrey's divergence, etc.). In contrast, we propose to use the intrinsic Riemannian distance on the manifold of SPD matrices. Our main contribution is a novel mathematical model for sparse coding of SPD matrices; we also present a computationally simple algorithm for optimizing our model. Experiments on several computer vision datasets showcase superior classification and retrieval performance compared with state-of-the-art approaches
Guaranteed clustering and biclustering via semidefinite programming
Identifying clusters of similar objects in data plays a significant role in a
wide range of applications. As a model problem for clustering, we consider the
densest k-disjoint-clique problem, whose goal is to identify the collection of
k disjoint cliques of a given weighted complete graph maximizing the sum of the
densities of the complete subgraphs induced by these cliques. In this paper, we
establish conditions ensuring exact recovery of the densest k cliques of a
given graph from the optimal solution of a particular semidefinite program. In
particular, the semidefinite relaxation is exact for input graphs corresponding
to data consisting of k large, distinct clusters and a smaller number of
outliers. This approach also yields a semidefinite relaxation for the
biclustering problem with similar recovery guarantees. Given a set of objects
and a set of features exhibited by these objects, biclustering seeks to
simultaneously group the objects and features according to their expression
levels. This problem may be posed as partitioning the nodes of a weighted
bipartite complete graph such that the sum of the densities of the resulting
bipartite complete subgraphs is maximized. As in our analysis of the densest
k-disjoint-clique problem, we show that the correct partition of the objects
and features can be recovered from the optimal solution of a semidefinite
program in the case that the given data consists of several disjoint sets of
objects exhibiting similar features. Empirical evidence from numerical
experiments supporting these theoretical guarantees is also provided
Implementation of an Optimal First-Order Method for Strongly Convex Total Variation Regularization
We present a practical implementation of an optimal first-order method, due
to Nesterov, for large-scale total variation regularization in tomographic
reconstruction, image deblurring, etc. The algorithm applies to -strongly
convex objective functions with -Lipschitz continuous gradient. In the
framework of Nesterov both and are assumed known -- an assumption
that is seldom satisfied in practice. We propose to incorporate mechanisms to
estimate locally sufficient and during the iterations. The mechanisms
also allow for the application to non-strongly convex functions. We discuss the
iteration complexity of several first-order methods, including the proposed
algorithm, and we use a 3D tomography problem to compare the performance of
these methods. The results show that for ill-conditioned problems solved to
high accuracy, the proposed method significantly outperforms state-of-the-art
first-order methods, as also suggested by theoretical results.Comment: 23 pages, 4 figure
Filter-based DIRECT method for constrained global optimization
This paper presents a DIRECT-type method that uses a filter methodology to assure convergence to a feasible and optimal solution of nonsmooth and nonconvex constrained global optimization problems. The filter methodology aims to give priority to the selection of hyperrectangles with feasible center points, followed by those with infeasible and non-dominated center points and finally by those that have infeasible and dominated center points. The convergence properties of the algorithm are analyzed. Preliminary numerical experiments show that the proposed filter-based DIRECT algorithm gives competitive results when compared with other DIRECT-type methods.The authors would like to thank two anonymous referees and the Associate Editor for their
valuable comments and suggestions to improve the paper.
This work has been supported by COMPETE: POCI-01-0145-FEDER-007043 and FCT
- Fundac¸ao para a Ciência e Tecnologia within the projects UID/CEC/00319/2013 and ˆ
UID/MAT/00013/2013.info:eu-repo/semantics/publishedVersio
Accelerating gradient projection methods for -constrained signal recovery by steplength selection rules
We propose a new gradient projection algorithm that compares favorably with
the fastest algorithms available to date for -constrained sparse
recovery from noisy data, both in the compressed sensing and inverse problem
frameworks. The method exploits a line-search along the feasible direction and
an adaptive steplength selection based on recent strategies for the alternation
of the well-known Barzilai-Borwein rules. The convergence of the proposed
approach is discussed and a computational study on both well-conditioned and
ill-conditioned problems is carried out for performance evaluations in
comparison with five other algorithms proposed in the literature.Comment: 11 pages, 4 figure
Combining filter method and dynamically dimensioned search for constrained global optimization
In this work we present an algorithm that combines the filter technique and the dynamically dimensioned search (DDS) for solving nonlinear and nonconvex constrained global optimization problems. The DDS is a stochastic global algorithm for solving bound constrained problems that in each iteration generates a randomly trial point perturbing some coordinates of the current best point. The filter technique controls the progress related to optimality and feasibility defining a forbidden region of points refused by the algorithm. This region can be given by the flat or slanting filter rule. The proposed algorithm does not compute or approximate any derivatives of the objective and constraint functions. Preliminary experiments show that the proposed algorithm gives competitive results when compared with other methods.The first author thanks a scholarship supported by the International
Cooperation Program CAPES/ COFECUB at the University of Minho.
The second and third authors thanks the support given by FCT (Funda¸c˜ao para
Ciˆencia e Tecnologia, Portugal) in the scope of the projects: UID/MAT/00013/2013
and UID/CEC/00319/2013. The fourth author was partially supported by CNPq-Brazil
grants 308957/2014-8 and 401288/2014-5.info:eu-repo/semantics/publishedVersio
An artificial fish swarm filter-based Method for constrained global optimization
Ana Maria A.C. Rocha, M. Fernanda P. Costa and Edite M.G.P. Fernandes, An Artificial Fish Swarm Filter-Based Method for Constrained Global Optimization, B. Murgante, O. Gervasi, S. Mirsa, N. Nedjah, A.M. Rocha, D. Taniar, B. Apduhan (Eds.), Lecture Notes in Computer Science, Part III, LNCS 7335, pp. 57–71, Springer, Heidelberg, 2012.An artificial fish swarm algorithm based on a filter methodology
for trial solutions acceptance is analyzed for general constrained
global optimization problems. The new method uses the filter set concept
to accept, at each iteration, a population of trial solutions whenever
they improve constraint violation or objective function, relative to the
current solutions. The preliminary numerical experiments with a wellknown
benchmark set of engineering design problems show the effectiveness
of the proposed method.Fundação para a Ciência e a Tecnologia (FCT
Filter-based stochastic algorithm for global optimization
We propose the general Filter-based Stochastic Algorithm (FbSA) for the global optimization of nonconvex and nonsmooth constrained problems. Under certain conditions on the probability distributions that generate the sample points, almost sure convergence is proved. In order to optimize problems with computationally expensive black-box objective functions, we develop the FbSA-RBF algorithm based on the general FbSA and assisted by Radial Basis Function (RBF) surrogate models to approximate the objective function. At each iteration, the resulting algorithm constructs/updates a surrogate model of the objective function and generates trial points using a dynamic coordinate search strategy similar to the one used in the Dynamically Dimensioned Search method. To identify a promising best trial point, a non-dominance concept based on the values of the surrogate model and the constraint violation at the trial points is used. Theoretical results concerning the sufficient conditions for the almost surely convergence of the algorithm are presented. Preliminary numerical experiments show that the FbSA-RBF is competitive when compared with other known methods in the literature.The authors are grateful to the anonymous referees for their fruitful comments and suggestions.The first and second authors were partially supported by Brazilian Funds through CAPES andCNPq by Grants PDSE 99999.009400/2014-01 and 309303/2017-6. The research of the thirdand fourth authors were partially financed by Portuguese Funds through FCT (Fundação para Ciência e Tecnologia) within the Projects UIDB/00013/2020 and UIDP/00013/2020 of CMAT-UM and UIDB/00319/2020
- …