86 research outputs found
Efficient Solution of Portfolio Optimization Problems via Dimension Reduction and Sparsification
The Markowitz mean-variance portfolio optimization model aims to balance
expected return and risk when investing. However, there is a significant
limitation when solving large portfolio optimization problems efficiently: the
large and dense covariance matrix. Since portfolio performance can be
potentially improved by considering a wider range of investments, it is
imperative to be able to solve large portfolio optimization problems
efficiently, typically in microseconds. We propose dimension reduction and
increased sparsity as remedies for the covariance matrix. The size reduction is
based on predictions from machine learning techniques and the solution to a
linear programming problem. We find that using the efficient frontier from the
linear formulation is much better at predicting the assets on the Markowitz
efficient frontier, compared to the predictions from neural networks. Reducing
the covariance matrix based on these predictions decreases both runtime and
total iterations. We also present a technique to sparsify the covariance matrix
such that it preserves positive semi-definiteness, which improves runtime per
iteration. The methods we discuss all achieved similar portfolio expected risk
and return as we would obtain from a full dense covariance matrix but with
improved optimizer performance.Comment: 14 pages, 3 figure
Efficient online subspace learning with an indefinite kernel for visual tracking and recognition
We propose an exact framework for online learning with a family of indefinite (not positive) kernels. As we study the case of nonpositive kernels, we first show how to extend kernel principal component analysis (KPCA) from a reproducing kernel Hilbert space to Krein space. We then formulate an incremental KPCA in Krein space that does not require the calculation of preimages and therefore is both efficient and exact. Our approach has been motivated by the application of visual tracking for which we wish to employ a robust gradient-based kernel. We use the proposed nonlinear appearance model learned online via KPCA in Krein space for visual tracking in many popular and difficult tracking scenarios. We also show applications of our kernel framework for the problem of face recognition
Parallel Sampling-Pipeline for Indefinite Stream of Heterogeneous Graphs using OpenCL for FPGAs
In the field of data science, a huge amount of data, generally represented as graphs, needs to be processed and analyzed. It is of utmost importance that this data be processed swiftly and efficiently to save time and energy. The volume and velocity of data, along with irregular access patterns in graph data structures, pose challenges in terms of analysis and processing. Further, a big chunk of time and energy is spent on analyzing these graphs on large compute clusters and/or data-centers. Filtering and refining of data using graph sampling techniques are one of the most effective ways to speed up the analysis. Efficient accelerators, such as FPGAs, have proven to significantly lower the energy cost of running an algorithm. To this end, we present the design and implementation of a parallel graph sampling technique, for a large number of input graphs streaming into a FPGA. A parallel approach using OpenCL for FPGAs was adopted to come up with a solution that is both time- and energyefficient. We introduce a novel graph data structure, suitable for streaming graphs on FPGAs, that allows time- and memory-efficient representation of graphs. Our experiments show that our proposed technique is 3x faster and 2x more energy efficient as compared to serial CPU version of the algorithm
Conservative Sparsification for Efficient Approximate Estimation
Linear Gaussian systems often exhibit sparse structures. For systems which grow as a function of time, marginalisation of past states will eventually introduce extra non-zero elements into the information matrix of the Gaussian distribution. These extra non-zeros can lead to dense problems as these systems progress through time. This thesis proposes a method that can delete elements of the information matrix while maintaining guarantees about the conservativeness of the resulting estimate with a computational complexity that is a function of the connectivity of the graph rather than the problem dimension. This sparsification can be performed iteratively and minimises the Kullback Leibler Divergence (KLD) between the original and approximate distributions. This new technique is called Conservative Sparsification (CS). For large sparse graphs employing a Junction Tree (JT) for estimation, efficiency is related to the size of the largest clique. Conservative Sparsification can be applied to clique splitting in JTs, enabling approximate and efficient estimation in JTs with the same conservative guarantees as CS for information matrices. In distributed estimation scenarios which use JTs, CS can be performed in parallel and asynchronously on JT cliques. This approach usually results in a larger KLD compared with the optimal CS approach, but an upper bound on this increased divergence can be calculated with information locally available to each clique. This work has applications in large scale distributed linear estimation problems where the size of the problem or communication overheads make optimal linear estimation difficult
A regularized Interior Point Method for sparse Optimal Transport on Graphs
In this work, the authors address the Optimal Transport (OT) problem on
graphs using a proximal stabilized Interior Point Method (IPM). In particular,
strongly leveraging on the induced primal-dual regularization, the authors
propose to solve large scale OT problems on sparse graphs using a bespoke IPM
algorithm able to suitably exploit primal-dual regularization in order to
enforce scalability. Indeed, the authors prove that the introduction of the
regularization allows to use sparsified versions of the normal Newton equations
to inexpensively generate IPM search directions. A detailed theoretical
analysis is carried out showing the polynomial convergence of the inner
algorithm in the proposed computational framework. Moreover, the presented
numerical results showcase the efficiency and robustness of the proposed
approach when compared to network simplex solvers
Graph coarsening: From scientific computing to machine learning
The general method of graph coarsening or graph reduction has been a
remarkably useful and ubiquitous tool in scientific computing and it is now
just starting to have a similar impact in machine learning. The goal of this
paper is to take a broad look into coarsening techniques that have been
successfully deployed in scientific computing and see how similar principles
are finding their way in more recent applications related to machine learning.
In scientific computing, coarsening plays a central role in algebraic multigrid
methods as well as the related class of multilevel incomplete LU
factorizations. In machine learning, graph coarsening goes under various names,
e.g., graph downsampling or graph reduction. Its goal in most cases is to
replace some original graph by one which has fewer nodes, but whose structure
and characteristics are similar to those of the original graph. As will be
seen, a common strategy in these methods is to rely on spectral properties to
define the coarse graph
Large-area visually augmented navigation for autonomous underwater vehicles
Submitted to the Joint Program in Applied Ocean Science & Engineering
in partial fulfillment of the requirements for the degree of Doctor of Philosophy
at the Massachusetts Institute of Technology
and the Woods Hole Oceanographic Institution
June 2005This thesis describes a vision-based, large-area, simultaneous localization and mapping (SLAM) algorithm that respects the low-overlap imagery constraints typical of autonomous underwater vehicles (AUVs) while exploiting the inertial sensor information that is routinely available on such platforms. We adopt a systems-level approach exploiting the complementary aspects of inertial sensing and visual perception from a calibrated pose-instrumented platform. This systems-level strategy yields a robust solution to underwater imaging that
overcomes many of the unique challenges of a marine environment (e.g., unstructured terrain, low-overlap imagery, moving light source). Our large-area SLAM algorithm recursively incorporates relative-pose constraints using a view-based representation that exploits exact sparsity in the Gaussian canonical form. This sparsity allows for efficient O(n) update complexity in the number of images composing the view-based map by utilizing recent multilevel relaxation techniques. We show that our algorithmic formulation is inherently sparse unlike other feature-based canonical SLAM algorithms, which impose sparseness via pruning approximations. In particular, we investigate
the sparsification methodology employed by sparse extended information filters (SEIFs)
and offer new insight as to why, and how, its approximation can lead to inconsistencies in
the estimated state errors. Lastly, we present a novel algorithm for efficiently extracting consistent marginal covariances useful for data association from the information matrix. In summary, this thesis advances the current state-of-the-art in underwater visual navigation by demonstrating end-to-end automatic processing of the largest visually navigated dataset to date using data collected from a survey of the RMS Titanic (path length over 3 km and 3100 m2 of mapped area). This accomplishment embodies the summed contributions of this thesis to several current SLAM research issues including scalability, 6 degree of
freedom motion, unstructured environments, and visual perception.This work was funded in part by the CenSSIS ERC of the National Science Foundation
under grant EEC-9986821, in part by the Woods Hole Oceanographic Institution through a
grant from the Penzance Foundation, and in part by a NDSEG Fellowship awarded through
the Department of Defense
- …