509 research outputs found
Computational Methods for Support Vector Machine Classification and Large-Scale Kalman Filtering
The first half of this dissertation focuses on computational methods for solving the constrained quadratic program (QP) within the support vector machine (SVM) classifier. One of the SVM formulations requires the solution of bound and equality constrained QPs. We begin by describing an augmented Lagrangian approach which incorporates the equality constraint into the objective function, resulting in a bound constrained QP. Furthermore, all constraints may be incorporated into the objective function to yield an unconstrained quadratic program, allowing us to apply the conjugate gradient (CG) method. Lastly, we adapt the scaled gradient projection method of [10] to the SVM QP and compare the performance of these methods with the state-of-the-art sequential minimal optimization algorithm and MATLAB\u27s built in constrained QP solver, quadprog. The augmented Lagrangian method outperforms other state-of-the-art methods on three image test cases. The second half of this dissertation focuses on computational methods for large-scale Kalman filtering applications. The Kalman filter (KF) is a method for solving a dynamic, coupled system of equations. While these methods require only linear algebra, standard KF is often infeasible in large-scale implementations due to the storage requirements and inverse calculations of large, dense covariance matrices. We introduce the use of the CG and Lanczos methods into various forms of the Kalman filter for low-rank approximations of the covariance matrices, with low-storage requirements. We also use CG for efficient Gaussian sampling within the ensemble Kalman filter method. The CG-based KF methods perform similarly in root-mean-square error when compared to the standard KF methods, when the standard implementations are feasible, and outperform the limited-memory Broyden-Fletcher-Goldfarb-Shanno approximation method
Factor Models with Real Data: a Robust Estimation of the Number of Factors
Factor models are a very efficient way to describe high dimensional vectors
of data in terms of a small number of common relevant factors. This problem,
which is of fundamental importance in many disciplines, is usually reformulated
in mathematical terms as follows. We are given the covariance matrix Sigma of
the available data. Sigma must be additively decomposed as the sum of two
positive semidefinite matrices D and L: D | that accounts for the idiosyncratic
noise affecting the knowledge of each component of the available vector of data
| must be diagonal and L must have the smallest possible rank in order to
describe the available data in terms of the smallest possible number of
independent factors.
In practice, however, the matrix Sigma is never known and therefore it must
be estimated from the data so that only an approximation of Sigma is actually
available. This paper discusses the issues that arise from this uncertainty and
provides a strategy to deal with the problem of robustly estimating the number
of factors.Comment: arXiv admin note: text overlap with arXiv:1708.0040
Nonconvex Generalization of ADMM for Nonlinear Equality Constrained Problems
The ever-increasing demand for efficient and distributed optimization
algorithms for large-scale data has led to the growing popularity of the
Alternating Direction Method of Multipliers (ADMM). However, although the use
of ADMM to solve linear equality constrained problems is well understood, we
lacks a generic framework for solving problems with nonlinear equality
constraints, which are common in practical applications (e.g., spherical
constraints). To address this problem, we are proposing a new generic ADMM
framework for handling nonlinear equality constraints, neADMM. After
introducing the generalized problem formulation and the neADMM algorithm, the
convergence properties of neADMM are discussed, along with its sublinear
convergence rate , where is the number of iterations. Next, two
important applications of neADMM are considered and the paper concludes by
describing extensive experiments on several synthetic and real-world datasets
to demonstrate the convergence and effectiveness of neADMM compared to existing
state-of-the-art methods
Doctor of Philosophy
dissertationThis dissertation aims to develop an innovative and improved paradigm for real-time large-scale traffic system estimation and mobility optimization. To fully utilize heterogeneous data sources in a complex spatial environment, this dissertation proposes an integrated and unified estimation-optimization framework capable of interpreting different types of traffic measurements into various decision-making processes. With a particular emphasis on the end-to-end travel time prediction problem, this dissertation proposes an information-theoretic sensor location model that aims to maximize information gains from a set of point, point-to-point and probe sensors in a traffic network. After thoroughly examining a number of possible measures of information gain, this dissertation selects a path travel time prediction uncertainty criterion to construct a joint sensor location and travel time estimation/prediction framework. To better measure the quality of service for ransportation systems, this dissertation investigates the path travel time reliability from two perspectives: variability and robustness. Based on calibrated travel disutility functions, the path travel time variability in this research is represented by its standard deviation in addition to the mean travel time. To handle the nonlinear and nonadditive cost functions introduced by the quadratic forms of the standard deviation term, a novel Lagrangian substitution approach is introduced to estimate the lower bound of the most reliable path solution through solving a sequence of standard shortest path problems. To recognize the asymmetrical and heavy-tailed travel time distributions, this dissertation proposes Lagrangian relaxation based iterative search algorithms for finding the absolute and percentile robust shortest paths. Moreover, this research develops a sampling-based method to dynamically construct a proxy objective function in terms of travel time observations from multiple days. Comprehensive numerical experiment results with real-world travel time measurements show that 10-20 iterations of standard shortest path algorithms for the reformulated models can offer a very small relative duality gap of about 2-6%, for both reliability measure models. This broadly-defined research has successfully addressed a number of theoretically challenging and practically important issues for building the next-generation Advanced Traveler Information Systems, and is expected to offer a rich foundation beneficial to the model and algorithmic development of sensor network design, traffic forecasting and personalized navigation
- …