Sequential Logistic Principal Component Analysis (SLPCA): Dimensional Reduction in Streaming Multivariate Binary-State System

Sequential or online dimensional reduction is of interests due to the explosion of streaming data based applications and the requirement of adaptive statistical modeling, in many emerging fields, such as the modeling of energy end-use profile. Principal Component Analysis (PCA), is the classical way of dimensional reduction. However, traditional Singular Value Decomposition (SVD) based PCA fails to model data which largely deviates from Gaussian distribution. The Bregman Divergence was recently introduced to achieve a generalized PCA framework. If the random variable under dimensional reduction follows Bernoulli distribution, which occurs in many emerging fields, the generalized PCA is called Logistic PCA (LPCA). In this paper, we extend the batch LPCA to a sequential version (i.e. SLPCA), based on the sequential convex optimization theory. The convergence property of this algorithm is discussed compared to the batch version of LPCA (i.e. BLPCA), as well as its performance in reducing the dimension for multivariate binary-state systems. Its application in building energy end-use profile modeling is also investigated.Comment: 6 pages, 4 figures, conference submissio

The Wasteland of Random Supergravities

We show that in a general \cal{N} = 1 supergravity with N \gg 1 scalar fields, an exponentially small fraction of the de Sitter critical points are metastable vacua. Taking the superpotential and Kahler potential to be random functions, we construct a random matrix model for the Hessian matrix, which is well-approximated by the sum of a Wigner matrix and two Wishart matrices. We compute the eigenvalue spectrum analytically from the free convolution of the constituent spectra and find that in typical configurations, a significant fraction of the eigenvalues are negative. Building on the Tracy-Widom law governing fluctuations of extreme eigenvalues, we determine the probability P of a large fluctuation in which all the eigenvalues become positive. Strong eigenvalue repulsion makes this extremely unlikely: we find P \propto exp(-c N^p), with c, p being constants. For generic critical points we find p \approx 1.5, while for approximately-supersymmetric critical points, p \approx 1.3. Our results have significant implications for the counting of de Sitter vacua in string theory, but the number of vacua remains vast.Comment: 39 pages, 9 figures; v2: fixed typos, added refs and clarification

Stochastic modelling of the spatial spread of influenza in Germany

In geographical epidemiology, disease counts are typically available in discrete spatial units and at discrete time-points. For example, surveillance data on infectious diseases usually consists of weekly counts of new infections in pre-defined geographical areas. Similarly, but on a different time-scale, cancer registries typically report yearly incidence or mortality counts in administrative regions. A major methodological challenge lies in building realistic models for space-time interactions on discrete irregular spatial graphs. In this paper, we will discuss an observation-driven approach, where past observed counts in neighbouring areas enter directly as explanatory variables, in contrast to the parameter-driven approach through latent Gaussian Markov random fields (Rue and Held, 2005) with spatio-temporal structure. The main focus will lie on the demonstration of the spread of influenza in Germany, obtained through the design and simulation of a spatial extension of the classical SIR model (Hufnagel et al., 2004)

Fast, Exact and Multi-Scale Inference for Semantic Image Segmentation with Deep Gaussian CRFs

In this work we propose a structured prediction technique that combines the virtues of Gaussian Conditional Random Fields (G-CRF) with Deep Learning: (a) our structured prediction task has a unique global optimum that is obtained exactly from the solution of a linear system (b) the gradients of our model parameters are analytically computed using closed form expressions, in contrast to the memory-demanding contemporary deep structured prediction approaches that rely on back-propagation-through-time, (c) our pairwise terms do not have to be simple hand-crafted expressions, as in the line of works building on the DenseCRF, but can rather be `discovered' from data through deep architectures, and (d) out system can trained in an end-to-end manner. Building on standard tools from numerical analysis we develop very efficient algorithms for inference and learning, as well as a customized technique adapted to the semantic segmentation task. This efficiency allows us to explore more sophisticated architectures for structured prediction in deep learning: we introduce multi-resolution architectures to couple information across scales in a joint optimization framework, yielding systematic improvements. We demonstrate the utility of our approach on the challenging VOC PASCAL 2012 image segmentation benchmark, showing substantial improvements over strong baselines. We make all of our code and experiments available at {https://github.com/siddharthachandra/gcrf}Comment: Our code is available at https://github.com/siddharthachandra/gcr

Geoadditive survival models

Survival data often contain geographical or spatial information, such as the residence of individuals. We propose geoadditive survival models for analyzing spatial effects jointly with possibly nonlinear effects of other covariates. Within a unified Bayesian framework, our approach extends the classical Cox model to a more general multiplicative hazard rate model, augmenting the common linear predictor with a spatial component and nonparametric terms for nonlinear effects of time and metrical covariates. Markov random fields and penalized regression splines are used as basic building blocks. Inference is fully Bayesian and uses computationally efficient MCMC sampling schemes. Smoothing parameters are an integral part of the model and are estimated automatically. Perfomance is investigated through simulation studies. We apply our approach to data from a case study in London and Essex that aims to estimate the effect of area of residence and further covariates on waiting times to coronary artery bypass graft (CABG)

Random Splitting of Fluid Models: Ergodicity and Convergence

We introduce a family of stochastic models motivated by the study of nonequilibrium steady states of fluid equations. These models decompose the deterministic dynamics of interest into fundamental building blocks, i.e., minimal vector fields preserving some fundamental aspects of the original dynamics. Randomness is injected by sequentially following each vector field for a random amount of time. We show under general assumptions that these random dynamics possess a unique invariant measure and converge almost surely to the original, deterministic model in the small noise limit. We apply our construction to the Lorenz-96 equations, often used in studies of chaos and data assimilation, and Galerkin approximations of the 2D Euler and Navier-Stokes equations. An interesting feature of the models developed is that they apply directly to the conservative dynamics and not just those with excitation and dissipation

Sparse Nonparametric Graphical Models

We present some nonparametric methods for graphical modeling. In the discrete case, where the data are binary or drawn from a finite alphabet, Markov random fields are already essentially nonparametric, since the cliques can take only a finite number of values. Continuous data are different. The Gaussian graphical model is the standard parametric model for continuous data, but it makes distributional assumptions that are often unrealistic. We discuss two approaches to building more flexible graphical models. One allows arbitrary graphs and a nonparametric extension of the Gaussian; the other uses kernel density estimation and restricts the graphs to trees and forests. Examples of both methods are presented. We also discuss possible future research directions for nonparametric graphical modeling.Comment: Published in at http://dx.doi.org/10.1214/12-STS391 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org