Estimation of discrete choice models with hybrid stochastic adaptive batch size algorithms

The emergence of Big Data has enabled new research perspectives in the discrete choice community. While the techniques to estimate Machine Learning models on a massive amount of data are well established, these have not yet been fully explored for the estimation of statistical Discrete Choice Models based on the random utility framework. In this article, we provide new ways of dealing with large datasets in the context of Discrete Choice Models. We achieve this by proposing new efficient stochastic optimization algorithms and extensively testing them alongside existing approaches. We develop these algorithms based on three main contributions: the use of a stochastic Hessian, the modification of the batch size, and a change of optimization algorithm depending on the batch size. A comprehensive experimental comparison of fifteen optimization algorithms is conducted across ten benchmark Discrete Choice Model cases. The results indicate that the HAMABS algorithm, a hybrid adaptive batch size stochastic method, is the best performing algorithm across the optimization benchmarks. This algorithm speeds up the optimization time by a factor of 23 on the largest model compared to existing algorithms used in practice. The integration of the new algorithms in Discrete Choice Models estimation software will significantly reduce the time required for model estimation and therefore enable researchers and practitioners to explore new approaches for the specification of choice models.Comment: 43 page

Latent state estimation in a class of nonlinear systems

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The problem of estimating latent or unobserved states of a dynamical system from observed data is studied in this thesis. Approximate filtering methods for discrete time series for a class of nonlinear systems are considered, which, in turn, require sampling from a partially specified discrete distribution. A new algorithm is proposed to sample from partially specified discrete distribution, where the specification is in terms of the first few moments of the distribution. This algorithm generates deterministic sigma points and corresponding probability weights, which match exactly a specified mean vector, a specified covariance matrix, the average of specified marginal skewness and the average of specified marginal kurtosis. Both the deterministic particles and the probability weights are given in closed form and no numerical optimization is required. This algorithm is then used in approximate Bayesian filtering for generation of particles and the associated probability weights which propagate higher order moment information about latent states. This method is extended to generate random sigma points (or particles) and corresponding probability weights that match the same moments. The algorithm is also shown to be useful in scenario generation for financial optimization. For a variety of important distributions, the proposed moment-matching algorithm for generating particles is shown to lead to approximation which is very close to maximum entropy approximation. In a separate, but related contribution to the field of nonlinear state estimation, a closed-form linear minimum variance filter is derived for the systems with stochastic parameter uncertainties. The expressions for eigenvalues of the perturbed filter are derived for comparison with eigenvalues of the unperturbed Kalman filter. Moment-matching approximation is proposed for the nonlinear systems with multiplicative stochastic noise

Diffusion Approximations for Online Principal Component Estimation and Global Convergence

In this paper, we propose to adopt the diffusion approximation tools to study the dynamics of Oja's iteration which is an online stochastic gradient descent method for the principal component analysis. Oja's iteration maintains a running estimate of the true principal component from streaming data and enjoys less temporal and spatial complexities. We show that the Oja's iteration for the top eigenvector generates a continuous-state discrete-time Markov chain over the unit sphere. We characterize the Oja's iteration in three phases using diffusion approximation and weak convergence tools. Our three-phase analysis further provides a finite-sample error bound for the running estimate, which matches the minimax information lower bound for principal component analysis under the additional assumption of bounded samples.Comment: Appeared in NIPS 201

A tutorial on recursive models for analyzing and predicting path choice behavior

The problem at the heart of this tutorial consists in modeling the path choice behavior of network users. This problem has been extensively studied in transportation science, where it is known as the route choice problem. In this literature, individuals' choice of paths are typically predicted using discrete choice models. This article is a tutorial on a specific category of discrete choice models called recursive, and it makes three main contributions: First, for the purpose of assisting future research on route choice, we provide a comprehensive background on the problem, linking it to different fields including inverse optimization and inverse reinforcement learning. Second, we formally introduce the problem and the recursive modeling idea along with an overview of existing models, their properties and applications. Third, we extensively analyze illustrative examples from different angles so that a novice reader can gain intuition on the problem and the advantages provided by recursive models in comparison to path-based ones

A comprehensive literature classification of simulation optimisation methods

Simulation Optimization (SO) provides a structured approach to the system design and configuration when analytical expressions for input/output relationships are unavailable. Several excellent surveys have been written on this topic. Each survey concentrates on only few classification criteria. This paper presents a literature survey with all classification criteria on techniques for SO according to the problem of characteristics such as shape of the response surface (global as compared to local optimization), objective functions (single or multiple objectives) and parameter spaces (discrete or continuous parameters). The survey focuses specifically on the SO problem that involves single per-formance measureSimulation Optimization, classification methods, literature survey