113 research outputs found
Zero-Convex Functions, Perturbation Resilience, and Subgradient Projections for Feasibility-Seeking Methods
The convex feasibility problem (CFP) is at the core of the modeling of many
problems in various areas of science. Subgradient projection methods are
important tools for solving the CFP because they enable the use of subgradient
calculations instead of orthogonal projections onto the individual sets of the
problem. Working in a real Hilbert space, we show that the sequential
subgradient projection method is perturbation resilient. By this we mean that
under appropriate conditions the sequence generated by the method converges
weakly, and sometimes also strongly, to a point in the intersection of the
given subsets of the feasibility problem, despite certain perturbations which
are allowed in each iterative step. Unlike previous works on solving the convex
feasibility problem, the involved functions, which induce the feasibility
problem's subsets, need not be convex. Instead, we allow them to belong to a
wider and richer class of functions satisfying a weaker condition, that we call
"zero-convexity". This class, which is introduced and discussed here, holds a
promise to solve optimization problems in various areas, especially in
non-smooth and non-convex optimization. The relevance of this study to
approximate minimization and to the recent superiorization methodology for
constrained optimization is explained.Comment: Mathematical Programming Series A, accepted for publicatio
Robust and large-scale quasiconvex programming in structure-from-motion
Structure-from-Motion (SfM) is a cornerstone of computer vision. Briefly speaking,
SfM is the task of simultaneously estimating the poses of the cameras behind a set of images of a
scene, and the 3D coordinates of the points in the scene.
Often, the optimisation problems that underpin SfM do not have closed-form solutions, and finding
solutions via numerical schemes is necessary. An objective function, which measures the discrepancy
of a geometric object (e.g., camera poses, rotations, 3D coordi- nates) with a set of image
measurements, is to be minimised. Each image measurement gives rise to an error function. For
example, the reprojection error, which measures the distance between an observed image point and
the projection of a 3D point onto the image, is a commonly used error function.
An influential optimisation paradigm in SfM is the ℓ₀₀ paradigm, where the objective function takes
the form of the maximum of all individual error functions (e.g. individual reprojection errors of
scene points). The benefit of the ℓ₀₀ paradigm is that the objective function of many SfM
optimisation problems become quasiconvex, hence there is a unique minimum in the objective
function. The task of formulating and minimising quasiconvex objective functions is called
quasiconvex programming.
Although tremendous progress in SfM techniques under the ℓ₀₀ paradigm has been made, there are still
unsatisfactorily solved problems, specifically, problems associated with large-scale input data and
outliers in the data. This thesis describes novel techniques to
tackle these problems.
A major weakness of the ℓ₀₀ paradigm is its susceptibility to outliers. This thesis improves the
robustness of ℓ₀₀ solutions against outliers by employing the least median of squares (LMS)
criterion, which amounts to minimising the median error. In the context of triangulation, this
thesis proposes a locally convergent robust algorithm underpinned by a novel quasiconvex plane
sweep technique. Imposing the LMS criterion achieves significant outlier tolerance, and, at the
same time, some properties of quasiconvexity greatly simplify the process of solving the LMS
problem.
Approximation is a commonly used technique to tackle large-scale input data. This thesis introduces
the coreset technique to quasiconvex programming problems. The coreset technique aims find a
representative subset of the input data, such that solving the same problem on the subset yields a
solution that is within known bound of the optimal solution on the complete input set. In
particular, this thesis develops a coreset approximate algorithm to handle large-scale
triangulation tasks.
Another technique to handle large-scale input data is to break the optimisation into multiple
smaller sub-problems. Such a decomposition usually speeds up the overall optimisation process,
and alleviates the limitation on memory. This thesis develops a large-scale optimisation algorithm
for the known rotation problem (KRot). The proposed method decomposes the original quasiconvex
programming problem with potentially hundreds of thousands of parameters into multiple sub-problems
with only three parameters each. An efficient solver based on a novel minimum enclosing ball
technique is proposed to solve the sub-problems.Thesis (Ph.D.) (Research by Publication) -- University of Adelaide, School of Computer Science, 201
Beyond Convexity: Stochastic Quasi-Convex Optimization
Stochastic convex optimization is a basic and well studied primitive in
machine learning. It is well known that convex and Lipschitz functions can be
minimized efficiently using Stochastic Gradient Descent (SGD). The Normalized
Gradient Descent (NGD) algorithm, is an adaptation of Gradient Descent, which
updates according to the direction of the gradients, rather than the gradients
themselves. In this paper we analyze a stochastic version of NGD and prove its
convergence to a global minimum for a wider class of functions: we require the
functions to be quasi-convex and locally-Lipschitz. Quasi-convexity broadens
the con- cept of unimodality to multidimensions and allows for certain types of
saddle points, which are a known hurdle for first-order optimization methods
such as gradient descent. Locally-Lipschitz functions are only required to be
Lipschitz in a small region around the optimum. This assumption circumvents
gradient explosion, which is another known hurdle for gradient descent
variants. Interestingly, unlike the vanilla SGD algorithm, the stochastic
normalized gradient descent algorithm provably requires a minimal minibatch
size
New algorithmic developments in maximum consensus robust fitting
In many computer vision applications, the task of robustly estimating the set of parameters of
a geometric model is a fundamental problem. Despite the longstanding research efforts on robust
model fitting, there remains significant scope for investigation. For a large number of geometric
estimation tasks in computer vision, maximum consensus is the most popular robust fitting
criterion. This thesis makes several contributions in the algorithms for consensus maximization.
Randomized hypothesize-and-verify algorithms are arguably the most widely used class of
techniques for robust estimation thanks to their simplicity. Though efficient, these randomized
heuristic methods do not guarantee finding good maximum consensus estimates. To improve the
randomize algorithms, guided sampling approaches have been developed. These methods take
advantage of additional domain information, such as descriptor matching scores, to guide the
sampling process. Subsets of the data that are more likely to result in good estimates are prioritized
for consideration. However, these guided sampling approaches are ineffective when good
domain information is not available. This thesis tackles this shortcoming by proposing a new
guided sampling algorithm, which is based on the class of LP-type problems and Monte Carlo
Tree Search (MCTS). The proposed algorithm relies on a fundamental geometric arrangement
of the data to guide the sampling process. Specifically, we take advantage of the underlying tree
structure of the maximum consensus problem and apply MCTS to efficiently search the tree.
Empirical results show that the new guided sampling strategy outperforms traditional randomized
methods.
Consensus maximization also plays a key role in robust point set registration. A special case
is the registration of deformable shapes. If the surfaces have the same intrinsic shapes, their
deformations can be described accurately by a conformal model. The uniformization theorem
allows the shapes to be conformally mapped onto a canonical domain, wherein the shapes can be
aligned using a M¨obius transformation. The problem of correspondence-free M¨obius alignment
of two sets of noisy and partially overlapping point sets can be tackled as a maximum consensus
problem. Solving for the M¨obius transformation can be approached by randomized voting-type
methods which offers no guarantee of optimality. Local methods such as Iterative Closest Point
can be applied, but with the assumption that a good initialization is given or these techniques
may converge to a bad local minima. When a globally optimal solution is required, the literature
has so far considered only brute-force search. This thesis contributes a new branch-and-bound
algorithm that solves for the globally optimal M¨obius transformation much more efficiently.
So far, the consensus maximization problems are approached mainly by randomized algorithms,
which are efficient but offer no analytical convergence guarantee. On the other hand,
there exist exact algorithms that can solve the problem up to global optimality. The global methods,
however, are intractable in general due to the NP-hardness of the consensus maximization. To fill the gap between the two extremes, this thesis contributes two novel deterministic algorithms
to approximately optimize the maximum consensus criterion. The first method is based
on non-smooth penalization supported by a Frank-Wolfe-style optimization scheme, and another
algorithm is based on Alternating Direction Method of Multipliers (ADMM). Both of the
proposed methods are capable of handling the non-linear geometric residuals commonly used in
computer vision. As will be demonstrated, our proposed methods consistently outperform other
heuristics and approximate methods.Thesis (Ph.D.) (Research by Publication) -- University of Adelaide, School of Computer Science, 201
- …