646 research outputs found
Global rates of convergence for nonconvex optimization on manifolds
We consider the minimization of a cost function on a manifold using
Riemannian gradient descent and Riemannian trust regions (RTR). We focus on
satisfying necessary optimality conditions within a tolerance .
Specifically, we show that, under Lipschitz-type assumptions on the pullbacks
of to the tangent spaces of , both of these algorithms produce points
with Riemannian gradient smaller than in
iterations. Furthermore, RTR returns a point where also the Riemannian
Hessian's least eigenvalue is larger than in
iterations. There are no assumptions on initialization.
The rates match their (sharp) unconstrained counterparts as a function of the
accuracy (up to constants) and hence are sharp in that sense.
These are the first deterministic results for global rates of convergence to
approximate first- and second-order Karush-Kuhn-Tucker points on manifolds.
They apply in particular for optimization constrained to compact submanifolds
of , under simpler assumptions.Comment: 33 pages, IMA Journal of Numerical Analysis, 201
Cram\'er-Rao bounds for synchronization of rotations
Synchronization of rotations is the problem of estimating a set of rotations
R_i in SO(n), i = 1, ..., N, based on noisy measurements of relative rotations
R_i R_j^T. This fundamental problem has found many recent applications, most
importantly in structural biology. We provide a framework to study
synchronization as estimation on Riemannian manifolds for arbitrary n under a
large family of noise models. The noise models we address encompass zero-mean
isotropic noise, and we develop tools for Gaussian-like as well as heavy-tail
types of noise in particular. As a main contribution, we derive the
Cram\'er-Rao bounds of synchronization, that is, lower-bounds on the variance
of unbiased estimators. We find that these bounds are structured by the
pseudoinverse of the measurement graph Laplacian, where edge weights are
proportional to measurement quality. We leverage this to provide interpretation
in terms of random walks and visualization tools for these bounds in both the
anchored and anchor-free scenarios. Similar bounds previously established were
limited to rotations in the plane and Gaussian-like noise
A Riemannian rank-adaptive method for low-rank matrix completion
The low-rank matrix completion problem can be solved by Riemannian
optimization on a fixed-rank manifold. However, a drawback of the known
approaches is that the rank parameter has to be fixed a priori. In this paper,
we consider the optimization problem on the set of bounded-rank matrices. We
propose a Riemannian rank-adaptive method, which consists of fixed-rank
optimization, rank increase step and rank reduction step. We explore its
performance applied to the low-rank matrix completion problem. Numerical
experiments on synthetic and real-world datasets illustrate that the proposed
rank-adaptive method compares favorably with state-of-the-art algorithms. In
addition, it shows that one can incorporate each aspect of this rank-adaptive
framework separately into existing algorithms for the purpose of improving
performance.Comment: 22 pages, 12 figures, 1 tabl
Two Algorithms for Orthogonal Nonnegative Matrix Factorization with Application to Clustering
Approximate matrix factorization techniques with both nonnegativity and
orthogonality constraints, referred to as orthogonal nonnegative matrix
factorization (ONMF), have been recently introduced and shown to work
remarkably well for clustering tasks such as document classification. In this
paper, we introduce two new methods to solve ONMF. First, we show athematical
equivalence between ONMF and a weighted variant of spherical k-means, from
which we derive our first method, a simple EM-like algorithm. This also allows
us to determine when ONMF should be preferred to k-means and spherical k-means.
Our second method is based on an augmented Lagrangian approach. Standard ONMF
algorithms typically enforce nonnegativity for their iterates while trying to
achieve orthogonality at the limit (e.g., using a proper penalization term or a
suitably chosen search direction). Our method works the opposite way:
orthogonality is strictly imposed at each step while nonnegativity is
asymptotically obtained, using a quadratic penalty. Finally, we show that the
two proposed approaches compare favorably with standard ONMF algorithms on
synthetic, text and image data sets.Comment: 17 pages, 8 figures. New numerical experiments (document and
synthetic data sets
Searching for faint companions with VLTI/PIONIER. I. Method and first results
Context. A new four-telescope interferometric instrument called PIONIER has
recently been installed at VLTI. It provides improved imaging capabilities
together with high precision. Aims. We search for low-mass companions around a
few bright stars using different strategies, and determine the dynamic range
currently reachable with PIONIER. Methods. Our method is based on the closure
phase, which is the most robust interferometric quantity when searching for
faint companions. We computed the chi^2 goodness of fit for a series of binary
star models at different positions and with various flux ratios. The resulting
chi^2 cube was used to identify the best-fit binary model and evaluate its
significance, or to determine upper limits on the companion flux in case of non
detections. Results. No companion is found around Fomalhaut, tau Cet and
Regulus. The median upper limits at 3 sigma on the companion flux ratio are
respectively of 2.3e-3 (in 4 h), 3.5e-3 (in 3 h) and 5.4e-3 (in 1.5 h) on the
search region extending from 5 to 100 mas. Our observations confirm that the
previously detected near-infrared excess emissions around Fomalhaut and tau Cet
are not related to a low-mass companion, and instead come from an extended
source such as an exozodiacal disk. In the case of del Aqr, in 30 min of
observation, we obtain the first direct detection of a previously known
companion, at an angular distance of about 40 mas and with a flux ratio of
2.05e-2 \pm 0.16e-2. Due to the limited u,v plane coverage, its position can,
however, not be unambiguously determined. Conclusions. After only a few months
of operation, PIONIER has already achieved one of the best dynamic ranges
world-wide for multi-aperture interferometers. A dynamic range up to about
1:500 is demonstrated, but significant improvements are still required to reach
the ultimate goal of directly detecting hot giant extrasolar planets.Comment: 11 pages, 6 figures, accepted for publication in A&
Quadproj: a Python package for projecting onto quadratic hypersurfaces
Quadratic hypersurfaces are a natural generalization of affine subspaces, and
projections are elementary blocks of algorithms in optimization and machine
learning. It is therefore intriguing that no proper studies and tools have been
developed to tackle this nonconvex optimization problem. The quadproj package
is a user-friendly and documented software that is dedicated to project a point
onto a non-cylindrical central quadratic hypersurface
- …