Subspace identification via convex optimization

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2011.Cataloged from PDF version of thesis.Includes bibliographical references (p. 88-92).In this thesis we consider convex optimization-based approaches to the classical problem of identifying a subspace from noisy measurements of a random process taking values in the subspace. We focus on the case where the measurement noise is component-wise independent, known as the factor analysis model in statistics. We develop a new analysis of an existing convex optimization-based heuristic for this problem. Our analysis indicates that in high-dimensional settings, where both the ambient dimension and the dimension of the subspace to be identified are large, the convex heuristic, minimum trace factor analysis, is often very successful. We provide simple deterministic conditions on the underlying 'true' subspace under which the convex heuristic provably identifies the correct subspace. We also consider the performance of minimum trace factor analysis on 'typical' subspace identification problems, that is problems where the underlying subspace is chosen randomly from subspaces of a particular dimension. In this setting we establish conditions on the ambient dimension and the dimension of the underlying subspace under which the convex heuristic identifies the subspace correctly with high probability. We then consider a refinement of the subspace identification problem where we aim to identify a class of structured subspaces arising from Gaussian latent tree models. More precisely, given the covariance at the finest scale of a Gaussian latent tree model, and the tree that indexes the model, we aim to learn the parameters of the model, including the state dimensions of each of the latent variables. We do so by extending the convex heuristic, and our analysis, from the factor analysis setting to the setting of Gaussian latent tree models. We again provide deterministic conditions on the underlying latent tree model that ensure our convex optimization-based heuristic successfully identifies the parameters and state dimensions of the model.by James Saunderson.S.M

Applications of and extensions to state-space models

State-space models have proven invaluable in the analysis of dynamic data, specifically time series data. They provide a natural and interpretable framework to learn about and describe dynamic processes. State-space models also provide a flexible framework for embedding prescriptive, mathematical models in ways that account for multiple sources of uncertainty. When considered within the more general directed graphical model formalism, state-space models can be reimagined and extended into arenas beyond the reach of traditional state-space models. In three papers, we consider various applications of and extensions to state-space models. All papers stem from collaborations with Los Alamos National Laboratory. In the field of space weather forecasting, many modeling approaches have been developed in the last 25 years. These approaches attempt to make sense of the dynamic and not-well-understood relationships between electron flux intensities and relevant covariates. Many of these forecasting models possess inherent limitations because they are static in nature and thus are constrained to customized and narrow time windows. In Chapter 2, we discuss these limitations and present an alternate approach to space weather forecasting utilizing dynamic linear models (DLMs). Benefits of dynamic modeling when compared to static modeling are discussed and ground work is laid for future dynamic forecasting endeavors in space weather. This work was published in the journal Space Weather under research article number 10.1002/2014SW001057 in June 2014. Multiscale modeling involves decomposing or explicitly modeling processes that arise at multiple scales. In Chapter 3, we extend the DLM into the multiscale arena with the presentation of the multiscale dynamic linear model (MSDLM). We present the MSDLM within the directed graphical model formalism. In so doing, we provide the necessary background to consider multiscale modeling generally. The MSDLM is a multiscale time series model that is interpretable, flexible, and coherently combines multiple scales of information in a principled, unified framework. Estimation and sampling procedures are presented. We illustrate the efficacy of the MSDLM by revisiting the problem of space weather forecasting discussed in Chapter 2. Forecasting seasonal influenza in the U.S. is challenging and consequential. It is challenging because there is uncertainty in the form of the disease transmission process, the process is only partially observed, and those observations are noisy. It is consequential because influenza poses serious risks to both national security and public health. In Chapter 4, we propose a non-Gaussian, nonlinear state-space model that embeds a compartmental model (i.e., a set of nonlinear, ordinary differential equations) into the state equations. The state-space framework provides valuable flexibility to the deterministic disease transmission process while simultaneously allowing and accounting for uncertainties in the parameters, the process, and the measurement mechanism. Prior specification is discussed in detail. Forecasting metrics are proposed and compared with competing models