Improved methods for statistical inference in the context of various types of parameter variation

This dissertation addresses various issues related to statistical inference in the context of parameter time-variation. The problem is considered within general regression models as well as in the context of methods for forecast evaluation. The first chapter develops a theory of evolutionary spectra for heteroskedasticityand autocorrelation-robust (HAR) inference when the data may not satisfy secondorder stationarity. We introduce a class of nonstationary stochastic processes that have a time-varying spectral representation and presents a new positive semidefinite heteroskedasticity- and autocorrelation consistent (HAC) estimator. We obtain an optimal HAC estimator under the mean-squared error (MSE) criterion and show its consistency. We propose a data-dependent procedure based on a “plug-in” approach that determines the bandwidth parameters for given kernels and a given sample size. The second chapter develops a continuous record asymptotic framework to build inference methods for the date of a structural change in a linear regression model. We impose very mild regularity conditions on an underlying continuous-time model assumed to generate the data. We consider the least-squares estimate of the break date and establish consistency and convergence rate. We provide a limit theory for shrinking magnitudes of shifts and locally increasing variances. The third chapter develops a novel continuous-time asymptotic framework for inference on whether the predictive ability of a given forecast model remains stable over time. As the sampling interval between observations shrinks to zero the sequence of forecast losses is approximated by a continuous-time stochastic process possessing certain pathwise properties. We consider an hypotheses testing problem based on the local properties of the continuous-time limit counterpart of the sequence of losses. The fourth chapter develops a class of Generalized Laplace (GL) inference methods for the change-point dates in a linear time series regression model with multiple structural changes. The GL estimator is defined by an integration rather than optimization-based method and relies on the least-squares criterion function. On the theoretical side, depending on some smoothing parameter, the class of GL estimators exhibits a dual limiting distribution; namely, the classical shrinkage asymptotic distribution of the least-squares estimator, or a Bayes-type asymptotic distribution

Automatic Backward Filtering Forward Guiding for Markov processes and graphical models

We incorporate discrete and continuous time Markov processes as building blocks into probabilistic graphical models with latent and observed variables. We introduce the automatic Backward Filtering Forward Guiding (BFFG) paradigm (Mider et al., 2020) for programmable inference on latent states and model parameters. Our starting point is a generative model, a forward description of the probabilistic process dynamics. We backpropagate the information provided by observations through the model to transform the generative (forward) model into a pre-conditional model guided by the data. It approximates the actual conditional model with known likelihood-ratio between the two. The backward filter and the forward change of measure are suitable to be incorporated into a probabilistic programming context because they can be formulated as a set of transformation rules. The guided generative model can be incorporated in different approaches to efficiently sample latent states and parameters conditional on observations. We show applicability in a variety of settings, including Markov chains with discrete state space, interacting particle systems, state space models, branching diffusions and Gamma processes

Online Discrimination of Nonlinear Dynamics with Switching Differential Equations

How to recognise whether an observed person walks or runs? We consider a dynamic environment where observations (e.g. the posture of a person) are caused by different dynamic processes (walking or running) which are active one at a time and which may transition from one to another at any time. For this setup, switching dynamic models have been suggested previously, mostly, for linear and nonlinear dynamics in discrete time. Motivated by basic principles of computations in the brain (dynamic, internal models) we suggest a model for switching nonlinear differential equations. The switching process in the model is implemented by a Hopfield network and we use parametric dynamic movement primitives to represent arbitrary rhythmic motions. The model generates observed dynamics by linearly interpolating the primitives weighted by the switching variables and it is constructed such that standard filtering algorithms can be applied. In two experiments with synthetic planar motion and a human motion capture data set we show that inference with the unscented Kalman filter can successfully discriminate several dynamic processes online

Continuous record Laplace-based inference about the break date in structural change models

Building upon the continuous record asymptotic framework recently introduced by Casini and Perron (2018a) for inference in structural change models, we propose a Laplace-based (Quasi-Bayes) procedure for the construction of the estimate and confidence set for the date of a structural change. It is defined by an integration rather than an optimization-based method.A transformation of the least-squares criterion function is evaluated in order to derive a proper distribution, referred to as the Quasi-posterior. For a given choice of a loss function, the Laplace-type estimator is the minimizer of the expected risk with the expectation taken under the Quasi-posterior. Besides providing an alternative estimate that is more precise—lower mean absolute error (MAE) and lower root-mean squared error (RMSE)—than the usual least-squares one, the Quasi-posterior distribution can be used to construct asymptotically valid inference using the concept of Highest Density Region. The resulting Laplace-based inferential procedure is shown to have lower MAE and RMSE, and the confidence sets strike the best balance between empirical coverage rates and average lengths of the confidence sets relative to traditional long-span methods, whether the break size is small or large.First author draf

Causal Inference in Discretely Observed Continuous Time Processes

In causal inference for longitudinal data, standard methods usually assume that the underlying processes are discrete time processes, and that the observational time points are the time points when the processes change values. The identification of these standard models often relies on the sequential randomization assumption, which assumes that the treatment assignment at each time point only depends on current covariates and the covariates and treatment that are observed in the past. However, in many real world data sets, it is more reasonable to assume that the underlying processes are continuous time processes, and that they are only observed at discrete time points. When this happens, the sequential randomization assumption may not be true even if it is still a reasonable abstraction of the treatment decision mechanism at the continuous time level. For example, in a multi-round survey study, the decision of treatment can be made by the subject and the subject\u27s physician in continuous time, while the treatment level and covariates are only collected in discrete times by a third party survey organization. The mismatch in the treatment decision time and the observational time makes the sequential randomization assumption false in the observed data. In this dissertation, we show that the standard methods could produce severely biased estimates, and we would explore what further assumptions need to be made to warrant the use of standard methods. If these assumptions are false, we advocate the use of controlling-the-future method of Joffe and Robins (2009) when we are able to reconstruct the potential outcomes from the discretely observed data. We propose a full modeling approach and demonstrate it by an example of estimating the effect of vitamin A deficiency on children\u27s respiratory infection, when we are not able to do so. We also provide a semi-parametric analysis of the controlling-the-future method, giving the semi-parametric efficient estimator

A Unifying review of linear gaussian models

Factor analysis, principal component analysis, mixtures of gaussian clusters, vector quantization, Kalman filter models, and hidden Markov models can all be unified as variations of unsupervised learning under a single basic generative model. This is achieved by collecting together disparate observations and derivations made by many previous authors and introducing a new way of linking discrete and continuous state models using a simple nonlinearity. Through the use of other nonlinearities, we show how independent component analysis is also a variation of the same basic generative model.We show that factor analysis and mixtures of gaussians can be implemented in autoencoder neural networks and learned using squared error plus the same regularization term. We introduce a new model for static data, known as sensible principal component analysis, as well as a novel concept of spatially adaptive observation noise. We also review some of the literature involving global and local mixtures of the basic models and provide pseudocode for inference and learning for all the basic models

Structure and Dynamics of Information Pathways in Online Media

Diffusion of information, spread of rumors and infectious diseases are all instances of stochastic processes that occur over the edges of an underlying network. Many times networks over which contagions spread are unobserved, and such networks are often dynamic and change over time. In this paper, we investigate the problem of inferring dynamic networks based on information diffusion data. We assume there is an unobserved dynamic network that changes over time, while we observe the results of a dynamic process spreading over the edges of the network. The task then is to infer the edges and the dynamics of the underlying network. We develop an on-line algorithm that relies on stochastic convex optimization to efficiently solve the dynamic network inference problem. We apply our algorithm to information diffusion among 3.3 million mainstream media and blog sites and experiment with more than 179 million different pieces of information spreading over the network in a one year period. We study the evolution of information pathways in the online media space and find interesting insights. Information pathways for general recurrent topics are more stable across time than for on-going news events. Clusters of news media sites and blogs often emerge and vanish in matter of days for on-going news events. Major social movements and events involving civil population, such as the Libyan's civil war or Syria's uprise, lead to an increased amount of information pathways among blogs as well as in the overall increase in the network centrality of blogs and social media sites.Comment: To Appear at the 6th International Conference on Web Search and Data Mining (WSDM '13