598 research outputs found
Statistical Inference for Structured High-dimensional Models
High-dimensional statistical inference is a newly emerged direction of statistical science in the 21 century. Its importance is due to the increasing dimensionality and complexity of models needed to process and understand the modern real world data. The main idea making possible meaningful inference about such models is to assume suitable lower dimensional underlying structure or low-dimensional approximations, for which the error can be reasonably controlled. Several types of such structures have been recently introduced including sparse high-dimensional regression, sparse and/or low rank matrix models, matrix completion models, dictionary learning, network models (stochastic block model, mixed membership models) and more. The workshop focused on recent developments in structured sequence and regression models, matrix and tensor estimation, robustness, statistical learning in complex settings, network data, and topic models
Graphon Estimation in bipartite graphs with observable edge labels and unobservable node labels
Many real-world data sets can be presented in the form of a matrix whose
entries correspond to the interaction between two entities of different natures
(number of times a web user visits a web page, a student's grade in a subject,
a patient's rating of a doctor, etc.). We assume in this paper that the
mentioned interaction is determined by unobservable latent variables describing
each entity. Our objective is to estimate the conditional expectation of the
data matrix given the unobservable variables. This is presented as a problem of
estimation of a bivariate function referred to as graphon. We study the cases
of piecewise constant and H\"older-continuous graphons. We establish finite
sample risk bounds for the least squares estimator and the exponentially
weighted aggregate. These bounds highlight the dependence of the estimation
error on the size of the data set, the maximum intensity of the interactions,
and the level of noise. As the analyzed least-squares estimator is intractable,
we propose an adaptation of Lloyd's alternating minimization algorithm to
compute an approximation of the least-squares estimator. Finally, we present
numerical experiments in order to illustrate the empirical performance of the
graphon estimator on synthetic data sets
Efficient posterior sampling for high-dimensional imbalanced logistic regression
High-dimensional data are routinely collected in many areas. We are
particularly interested in Bayesian classification models in which one or more
variables are imbalanced. Current Markov chain Monte Carlo algorithms for
posterior computation are inefficient as and/or increase due to
worsening time per step and mixing rates. One strategy is to use a
gradient-based sampler to improve mixing while using data sub-samples to reduce
per-step computational complexity. However, usual sub-sampling breaks down when
applied to imbalanced data. Instead, we generalize piece-wise deterministic
Markov chain Monte Carlo algorithms to include importance-weighted and
mini-batch sub-sampling. These approaches maintain the correct stationary
distribution with arbitrarily small sub-samples, and substantially outperform
current competitors. We provide theoretical support and illustrate gains in
simulated and real data applications.Comment: 4 figure
Variational Bayesian Inference with Stochastic Search
Mean-field variational inference is a method for approximate Bayesian
posterior inference. It approximates a full posterior distribution with a
factorized set of distributions by maximizing a lower bound on the marginal
likelihood. This requires the ability to integrate a sum of terms in the log
joint likelihood using this factorized distribution. Often not all integrals
are in closed form, which is typically handled by using a lower bound. We
present an alternative algorithm based on stochastic optimization that allows
for direct optimization of the variational lower bound. This method uses
control variates to reduce the variance of the stochastic search gradient, in
which existing lower bounds can play an important role. We demonstrate the
approach on two non-conjugate models: logistic regression and an approximation
to the HDP.Comment: Appears in Proceedings of the 29th International Conference on
Machine Learning (ICML 2012
Approximate inference methods in probabilistic machine learning and Bayesian statistics
This thesis develops new methods for efficient approximate inference in probabilistic models. Such models are routinely used in different fields, yet they remain computationally challenging as they involve high-dimensional integrals. We propose different approximate inference approaches addressing some challenges in probabilistic machine learning and Bayesian statistics. First, we present a Bayesian framework for genome-wide inference of DNA methylation levels and devise an efficient particle filtering and smoothing algorithm that can be used to identify differentially methylated regions between case and control groups. Second, we present a scalable inference approach for state space models by combining variational methods with sequential Monte Carlo sampling. The method is applied to self-exciting point process models that allow for flexible dynamics in the latent intensity function. Third, a new variational density motivated by copulas is developed. This new variational family can be beneficial compared with Gaussian approximations, as illustrated on examples with Bayesian neural networks. Lastly, we make some progress in a gradient-based adaptation of Hamiltonian Monte Carlo samplers by maximizing an approximation of the proposal entropy
- …