21,006 research outputs found
Guaranteed Rank Minimization via Singular Value Projection
Minimizing the rank of a matrix subject to affine constraints is a
fundamental problem with many important applications in machine learning and
statistics. In this paper we propose a simple and fast algorithm SVP (Singular
Value Projection) for rank minimization with affine constraints (ARMP) and show
that SVP recovers the minimum rank solution for affine constraints that satisfy
the "restricted isometry property" and show robustness of our method to noise.
Our results improve upon a recent breakthrough by Recht, Fazel and Parillo
(RFP07) and Lee and Bresler (LB09) in three significant ways:
1) our method (SVP) is significantly simpler to analyze and easier to
implement,
2) we give recovery guarantees under strictly weaker isometry assumptions
3) we give geometric convergence guarantees for SVP even in presense of noise
and, as demonstrated empirically, SVP is significantly faster on real-world and
synthetic problems.
In addition, we address the practically important problem of low-rank matrix
completion (MCP), which can be seen as a special case of ARMP. We empirically
demonstrate that our algorithm recovers low-rank incoherent matrices from an
almost optimal number of uniformly sampled entries. We make partial progress
towards proving exact recovery and provide some intuition for the strong
performance of SVP applied to matrix completion by showing a more restricted
isometry property. Our algorithm outperforms existing methods, such as those of
\cite{RFP07,CR08,CT09,CCS08,KOM09,LB09}, for ARMP and the matrix-completion
problem by an order of magnitude and is also significantly more robust to
noise.Comment: An earlier version of this paper was submitted to NIPS-2009 on June
5, 200
On convergence and optimality of maximum-likelihood APA
Affine projection algorithm (APA) is a well-known algorithm in adaptive
filtering applications such as audio echo cancellation. APA relies on three
parameters: (projection order), (step size) and
(regularization parameter). It is known that running APA for a fixed set of
parameters leads to a tradeoff between convergence speed and accuracy.
Therefore, various methods for adaptively setting the parameters have been
proposed in the literature. Inspired by maximum likelihood (ML) estimation, we
derive a new ML-based approach for adaptively setting the parameters of APA,
which we refer to as ML-APA. For memoryless Gaussian inputs, we fully
characterize the expected misalignment error of ML-APA as a function of
iteration number and show that it converges to zero as . We
further prove that the achieved error is asymptotically optimal. ML-APA updates
its estimate once every samples. We also propose incremental ML-APA
(IML-APA), which updates the coefficients at every time step and outperforms
ML-APA in our simulations results. Our simulation results verify the analytical
conclusions for memoryless inputs and show that the new algorithms also perform
well for strongly correlated input signals
Global Trajectory Optimisation : Can We Prune the Solution Space When Considering Deep Space Manoeuvres? [Final Report]
This document contains a report on the work done under the ESA/Ariadna study 06/4101 on the global optimization of space trajectories with multiple gravity assist (GA) and deep space manoeuvres (DSM). The study was performed by a joint team of scientists from the University of Reading and the University of Glasgow
An hybrid system approach to nonlinear optimal control problems
We consider a nonlinear ordinary differential equation and want to control
its behavior so that it reaches a target by minimizing a cost function. Our
approach is to use hybrid systems to solve this problem: the complex dynamic is
replaced by piecewise affine approximations which allow an analytical
resolution. The sequence of affine models then forms a sequence of states of a
hybrid automaton. Given a sequence of states, we introduce an hybrid
approximation of the nonlinear controllable domain and propose a new algorithm
computing a controllable, piecewise convex approximation. The same way the
nonlinear optimal control problem is replaced by an hybrid piecewise affine
one. Stating a hybrid maximum principle suitable to our hybrid model, we deduce
the global structure of the hybrid optimal control steering the system to the
target
Adaptive Ranking Based Constraint Handling for Explicitly Constrained Black-Box Optimization
A novel explicit constraint handling technique for the covariance matrix
adaptation evolution strategy (CMA-ES) is proposed. The proposed constraint
handling exhibits two invariance properties. One is the invariance to arbitrary
element-wise increasing transformation of the objective and constraint
functions. The other is the invariance to arbitrary affine transformation of
the search space. The proposed technique virtually transforms a constrained
optimization problem into an unconstrained optimization problem by considering
an adaptive weighted sum of the ranking of the objective function values and
the ranking of the constraint violations that are measured by the Mahalanobis
distance between each candidate solution to its projection onto the boundary of
the constraints. Simulation results are presented and show that the CMA-ES with
the proposed constraint handling exhibits the affine invariance and performs
similarly to the CMA-ES on unconstrained counterparts.Comment: 9 page
- …