Semiparametric Pseudo-Likelihood Estimation in Markov Random Fields

International audienceProbabilistic graphical models for continuous variables can be built out of either parametric or nonparametric conditional density estimators. While several research efforts have been focusing on parametric approaches (such as Gaussian models), kernel-based estimators are still the only viable and well-understood option for nonparametric density estimation. This paper develops a semiparametric estimator of probability density functions based on the nonparanormal transformation, which has been recently proposed for mapping arbitrarily distributed data samples onto normally distributed datasets. Pointwise and uniform consistency properties are established for the developed method. The resulting density model is then applied to pseudo-likelihood estimation in Markov random fields. An experimental evaluation on data distributed according to a variety of density functions indicates that such semiparametric Markov random field models significantly outperform both their Gaussian and kernel-based alternatives in terms of prediction accuracy

Model Order Reduction based on Proper Generalized Decomposition for the Propagation of Uncertainties in Structural Dynamics

International audienceA priori model reduction methods based on separated representations are introduced for the prediction of the low frequency response of uncertain structures within a parametric stochastic framework. The Proper Generalized Decomposition method is used to construct a quasi-optimal separated representation of the random solution at some frequency samples. At each frequency, an accurate representation of the solution is obtained on reduced bases of spatial functions and stochastic functions. An extraction of the deterministic bases allows for the generation of a global reduced basis yielding a reduced order model of the uncertain structure which appears to be accurate on the whole frequency band under study and for all values of input random parameters. This strategy can be seen as an alternative to traditional constructions of reduced order models in structural dynamics in the presence of parametric uncertainties. This reduced order model can then be used for further analyses such as the computation of the response at unresolved frequencies or the computation of more accurate stochastic approximations at some frequencies of interest. The dynamic response being highly nonlinear with respect to the input random parameters, a second level of separation of variables is introduced for the representation of functions of multiple random parameters, thus allowing the introduction of very fine approximations in each parametric dimension even when dealing with high parametric dimension

Semiparametric Pseudo-Likelihood Estimation in Markov Random Fields

International audienceProbabilistic graphical models for continuous variables can be built out of either parametric or nonparametric conditional density estimators. While several research efforts have been focusing on parametric approaches (such as Gaussian models), kernel-based estimators are still the only viable and well-understood option for nonparametric density estimation. This paper develops a semiparametric estimator of probability density functions based on the nonparanormal transformation, which has been recently proposed for mapping arbitrarily distributed data samples onto normally distributed datasets. Pointwise and uniform consistency properties are established for the developed method. The resulting density model is then applied to pseudo-likelihood estimation in Markov random fields. An experimental evaluation on data distributed according to a variety of density functions indicates that such semiparametric Markov random field models significantly outperform both their Gaussian and kernel-based alternatives in terms of prediction accuracy

Score Function Features for Discriminative Learning: Matrix and Tensor Framework

Feature learning forms the cornerstone for tackling challenging learning problems in domains such as speech, computer vision and natural language processing. In this paper, we consider a novel class of matrix and tensor-valued features, which can be pre-trained using unlabeled samples. We present efficient algorithms for extracting discriminative information, given these pre-trained features and labeled samples for any related task. Our class of features are based on higher-order score functions, which capture local variations in the probability density function of the input. We establish a theoretical framework to characterize the nature of discriminative information that can be extracted from score-function features, when used in conjunction with labeled samples. We employ efficient spectral decomposition algorithms (on matrices and tensors) for extracting discriminative components. The advantage of employing tensor-valued features is that we can extract richer discriminative information in the form of an overcomplete representations. Thus, we present a novel framework for employing generative models of the input for discriminative learning.Comment: 29 page