Essays on structural changes in high dimensional econometric models

This dissertation consists of three essays on estimating and testing structural changes in high dimensional econometrics models. These essays are based on three working papers joint with Prof. Badi Baltagi and Prof. Chihwa Kao. The first essay considers estimating the date of a single common change in the regression coefficients of a heterogeneous large N and large T panel data model with or without strong cross- sectional dependence. The second essay considers estimating a high dimensional factor model with an unknown number of latent factors and a single common change in the number of factors and/or factor loadings. The third essay considers estimating a high dimensional factor model with an unknown number of latent factors and multiple common changes in the number of factors and/or factor loadings, and also testing procedures to detect the presence and number of structural changes. The first essay studies the asymptotic properties of the least squares estimator of the common change point in large heterogeneous panel data models under various sets of conditions on the change magnitude and N-T ratio, allowing N and T to go to infinity jointly. Consistency and limiting distribution are established under general conditions. A general Hajek-Renyi inequality is introduced to calculate the order of the expectation of sup-type terms. Both weak and strong cross-sectional dependence are considered. In the former case the least squares estimator is consistent as the number of subjects tends to infinity while in the latter case a two step estimator is proposed and consistency can be recovered once estimated factors are used to control the cross-sectional dependence. The limiting distribution is derived allowing the error process to be serially dependent and heteroskedastic of unknown form, and inference can be made based on the simulated distribution. The second essay tackles the identification and estimation of a high dimensional factor model with unknown number of latent factors and a single common break in the number of factors and/or factor loadings. Since the factors are unobservable, the change point estimator is based on the second moments of the estimated pseudo factors. This essay shows that the estimation error of the proposed estimator is bounded in probability as N and T go to infinity jointly. This essay also shows that the proposed estimator has a high degree of robustness to misspecification of the number of pseudo factors. With the estimated change point plugged in, consistency of the estimated number of pre and post- break factors and convergence rate of the estimated pre and post-break factor space are then established under fairly general assumptions. Finite sample performance of the proposed estimators is investigated using Monte Carlo experiments. The third essay considers high dimensional factor models with multiple common structural changes. Based on the second moments of the estimated pseudo factors, both joint and sequential estimation of the change points are considered. The estimation error of both estimators is bounded in probability as the cross-sectional dimension N and the time dimension T go to infinity jointly. The measurement error contained in the estimated pseudo factors has no effect on the asymptotic properties of the estimated change points as N and T go to infinity jointly, and no N-T ratio condition is needed. The estimated change points are plugged in to estimate the number of factors and the factor space in each regime. Although the estimated change points are inconsistent, using them asymptotically has no effect on subsequent estimation. This essay also proposes (i) tests for the null of no change versus the alternative of l changes and (ii) tests for the null of l changes versus the alternative of l + 1 changes. These tests allow us to make inference on the presence and number of structural changes. Simulation results show good performance of the proposed estimation and testing procedures

Optimizing Data-Intensive Computing with Efficient Configuration Tuning

As the complexity of distributed analytics systems evolves over time, more configuration parameters get exposed for tuning. While these numerous parameters allow users more control over how their workloads are executed, this flexibility comes at a cost, since finding the right configurations for such systems in a cost-effective way becomes challenging. In practice, several factors contribute to the complexity of tuning the configuration of those systems: the large configuration space, the diversity of the served workloads (each workload possibly requiring a different resource allocation strategy to run optimally), and the dynamic characteristics of these systems’ environment (e.g., increase in input data size, changes in the allocation of resources). Paradoxically, existing solutions for workload tuning either assume static tuning environment or workloads that are inexpensive to run (i.e. requiring hundreds of execution samples). Recently, Bayesian Optimisation (BO) strategies have been applied as a solution to enable efficient autotuning. They build a probabilistic model incrementally to predict the impact of the parameters on performance using a small number of execution samples. The incrementally constructed BO model is used to guide the tuning process and accelerate convergence to a near-optimal configuration. Unfortunately, for distributed analytics systems, the configuration space is too large to construct a good model using traditional BO, which fails to provide quick convergence in high dimensional configuration space. I argue that cost-effective tuning strategies can only be developed when taking into account: the frequent changes that can happen in the analytics workload/environment, the amortization of tuning costs and how this influences tuning profitability, the high dimensionality of configuration space and the need to cater for diverse workloads. To tackle these challenges, I propose Tuneful, an efficient configuration tuning framework for such expensive to tune systems. It works efficiently both initially (when little data is available) as well as later (as more tuning knowledge is acquired). It starts with learning workload-specific influential parameters incrementally and tunes those only, then when more tuning knowledge becomes available, it detects similarity across workloads and utilizes multitask BO to share the tuning knowledge across similar workloads. I show how augmenting the BO approach with parameters’ significance and workload similarity characteristics enables an efficient configuration tuning in high dimensional configuration space. Over diverse analytics workloads, this significantly accelerates both configuration tuning and cost amortization, saving search time by 2.7-3.7X at median compared to the-state-of-the-art approaches

Finding High-Dimensional D-OptimalDesigns for Logistic Models via Differential Evolution

D-optimal designs are frequently used in controlled experiments to obtain the most accurateestimate of model parameters at minimal cost. Finding them can be a challenging task, especially whenthere are many factors in a nonlinear model. As the number of factors becomes large and interact withone another, there are many more variables to optimize and the D-optimal design problem becomes highdimensionaland non-separable. Consequently, premature convergence issues arise. Candidate solutions gettrapped in local optima and the classical gradient-based optimization approaches to search for the D-optimaldesigns rarely succeed. We propose a specially designed version of differential evolution (DE) which is arepresentative gradient-free optimization approach to solve such high-dimensional optimization problems.The proposed specially designed DE uses a new novelty-based mutation strategy to explore the variousregions in the search space. The exploration of the regions will be carried out differently from the previouslyexplored regions and the diversity of the population can be preserved. The proposed novelty-based mutationstrategy is collaborated with two common DE mutation strategies to balance exploration and exploitationat the early or medium stage of the evolution. Additionally, we adapt the control parameters of DE as theevolution proceeds. Using logistic models with several factors on various design spaces as examples, oursimulation results show our algorithm can find D-optimal designs efficiently and the algorithm outperformsits competitors. As an application, we apply our algorithm and re-design a 10-factor car refueling experimentwith discrete and continuous factors and selected pairwise interactions. Our proposed algorithm was able toconsistently outperform the other algorithms and find a more efficient D-optimal design for the problem

Sum-of-Squares approach to feedback control of laminar wake flows

A novel nonlinear feedback control design methodology for incompressible fluid flows aiming at the optimisation of long-time averages of flow quantities is presented. It applies to reduced-order finite-dimensional models of fluid flows, expressed as a set of first-order nonlinear ordinary differential equations with the right-hand side being a polynomial function in the state variables and in the controls. The key idea, first discussed in Chernyshenko et al. 2014, Philos. T. Roy. Soc. 372(2020), is that the difficulties of treating and optimising long-time averages of a cost are relaxed by using the upper/lower bounds of such averages as the objective function. In this setting, control design reduces to finding a feedback controller that optimises the bound, subject to a polynomial inequality constraint involving the cost function, the nonlinear system, the controller itself and a tunable polynomial function. A numerically tractable approach to the solution of such optimisation problems, based on Sum-of-Squares techniques and semidefinite programming, is proposed. To showcase the methodology, the mitigation of the fluctuation kinetic energy in the unsteady wake behind a circular cylinder in the laminar regime at Re=100, via controlled angular motions of the surface, is numerically investigated. A compact reduced-order model that resolves the long-term behaviour of the fluid flow and the effects of actuation, is derived using Proper Orthogonal Decomposition and Galerkin projection. In a full-information setting, feedback controllers are then designed to reduce the long-time average of the kinetic energy associated with the limit cycle. These controllers are then implemented in direct numerical simulations of the actuated flow. Control performance, energy efficiency, and physical control mechanisms identified are analysed. Key elements, implications and future work are discussed

Action-Conditional Video Prediction using Deep Networks in Atari Games

Motivated by vision-based reinforcement learning (RL) problems, in particular Atari games from the recent benchmark Aracade Learning Environment (ALE), we consider spatio-temporal prediction problems where future (image-)frames are dependent on control variables or actions as well as previous frames. While not composed of natural scenes, frames in Atari games are high-dimensional in size, can involve tens of objects with one or more objects being controlled by the actions directly and many other objects being influenced indirectly, can involve entry and departure of objects, and can involve deep partial observability. We propose and evaluate two deep neural network architectures that consist of encoding, action-conditional transformation, and decoding layers based on convolutional neural networks and recurrent neural networks. Experimental results show that the proposed architectures are able to generate visually-realistic frames that are also useful for control over approximately 100-step action-conditional futures in some games. To the best of our knowledge, this paper is the first to make and evaluate long-term predictions on high-dimensional video conditioned by control inputs.Comment: Published at NIPS 2015 (Advances in Neural Information Processing Systems 28