2,703 research outputs found
Gamma Processes, Stick-Breaking, and Variational Inference
While most Bayesian nonparametric models in machine learning have focused on
the Dirichlet process, the beta process, or their variants, the gamma process
has recently emerged as a useful nonparametric prior in its own right. Current
inference schemes for models involving the gamma process are restricted to
MCMC-based methods, which limits their scalability. In this paper, we present a
variational inference framework for models involving gamma process priors. Our
approach is based on a novel stick-breaking constructive definition of the
gamma process. We prove correctness of this stick-breaking process by using the
characterization of the gamma process as a completely random measure (CRM), and
we explicitly derive the rate measure of our construction using Poisson process
machinery. We also derive error bounds on the truncation of the infinite
process required for variational inference, similar to the truncation analyses
for other nonparametric models based on the Dirichlet and beta processes. Our
representation is then used to derive a variational inference algorithm for a
particular Bayesian nonparametric latent structure formulation known as the
infinite Gamma-Poisson model, where the latent variables are drawn from a gamma
process prior with Poisson likelihoods. Finally, we present results for our
algorithms on nonnegative matrix factorization tasks on document corpora, and
show that we compare favorably to both sampling-based techniques and
variational approaches based on beta-Bernoulli priors
Beta-Product Poisson-Dirichlet Processes
Time series data may exhibit clustering over time and, in a multiple time
series context, the clustering behavior may differ across the series. This
paper is motivated by the Bayesian non--parametric modeling of the dependence
between the clustering structures and the distributions of different time
series. We follow a Dirichlet process mixture approach and introduce a new
class of multivariate dependent Dirichlet processes (DDP). The proposed DDP are
represented in terms of vector of stick-breaking processes with dependent
weights. The weights are beta random vectors that determine different and
dependent clustering effects along the dimension of the DDP vector. We discuss
some theoretical properties and provide an efficient Monte Carlo Markov Chain
algorithm for posterior computation. The effectiveness of the method is
illustrated with a simulation study and an application to the United States and
the European Union industrial production indexes
Cluster and Feature Modeling from Combinatorial Stochastic Processes
One of the focal points of the modern literature on Bayesian nonparametrics
has been the problem of clustering, or partitioning, where each data point is
modeled as being associated with one and only one of some collection of groups
called clusters or partition blocks. Underlying these Bayesian nonparametric
models are a set of interrelated stochastic processes, most notably the
Dirichlet process and the Chinese restaurant process. In this paper we provide
a formal development of an analogous problem, called feature modeling, for
associating data points with arbitrary nonnegative integer numbers of groups,
now called features or topics. We review the existing combinatorial stochastic
process representations for the clustering problem and develop analogous
representations for the feature modeling problem. These representations include
the beta process and the Indian buffet process as well as new representations
that provide insight into the connections between these processes. We thereby
bring the same level of completeness to the treatment of Bayesian nonparametric
feature modeling that has previously been achieved for Bayesian nonparametric
clustering.Comment: Published in at http://dx.doi.org/10.1214/13-STS434 the Statistical
Science (http://www.imstat.org/sts/) by the Institute of Mathematical
Statistics (http://www.imstat.org
On Approximations of the Beta Process in Latent Feature Models
The beta process has recently been widely used as a nonparametric prior for
different models in machine learning, including latent feature models. In this
paper, we prove the asymptotic consistency of the finite dimensional
approximation of the beta process due to Paisley \& Carin (2009). In addition,
we derive an almost sure approximation of the beta process. This approximation
provides a direct method to efficiently simulate the beta process. A simulated
example, illustrating the work of the method and comparing its performance to
several existing algorithms, is also included.Comment: 25 page
- …