387 research outputs found
Augment-and-Conquer Negative Binomial Processes
By developing data augmentation methods unique to the negative binomial (NB)
distribution, we unite seemingly disjoint count and mixture models under the NB
process framework. We develop fundamental properties of the models and derive
efficient Gibbs sampling inference. We show that the gamma-NB process can be
reduced to the hierarchical Dirichlet process with normalization, highlighting
its unique theoretical, structural and computational advantages. A variety of
NB processes with distinct sharing mechanisms are constructed and applied to
topic modeling, with connections to existing algorithms, showing the importance
of inferring both the NB dispersion and probability parameters.Comment: Neural Information Processing Systems, NIPS 201
Generalized Bregman Divergence and Gradient of Mutual Information for Vector Poisson Channels
We investigate connections between information-theoretic and
estimation-theoretic quantities in vector Poisson channel models. In
particular, we generalize the gradient of mutual information with respect to
key system parameters from the scalar to the vector Poisson channel model. We
also propose, as another contribution, a generalization of the classical
Bregman divergence that offers a means to encapsulate under a unifying
framework the gradient of mutual information results for scalar and vector
Poisson and Gaussian channel models. The so-called generalized Bregman
divergence is also shown to exhibit various properties akin to the properties
of the classical version. The vector Poisson channel model is drawing
considerable attention in view of its application in various domains: as an
example, the availability of the gradient of mutual information can be used in
conjunction with gradient descent methods to effect compressive-sensing
projection designs in emerging X-ray and document classification applications
Zero-Truncated Poisson Tensor Factorization for Massive Binary Tensors
We present a scalable Bayesian model for low-rank factorization of massive
tensors with binary observations. The proposed model has the following key
properties: (1) in contrast to the models based on the logistic or probit
likelihood, using a zero-truncated Poisson likelihood for binary data allows
our model to scale up in the number of \emph{ones} in the tensor, which is
especially appealing for massive but sparse binary tensors; (2)
side-information in form of binary pairwise relationships (e.g., an adjacency
network) between objects in any tensor mode can also be leveraged, which can be
especially useful in "cold-start" settings; and (3) the model admits simple
Bayesian inference via batch, as well as \emph{online} MCMC; the latter allows
scaling up even for \emph{dense} binary data (i.e., when the number of ones in
the tensor/network is also massive). In addition, non-negative factor matrices
in our model provide easy interpretability, and the tensor rank can be inferred
from the data. We evaluate our model on several large-scale real-world binary
tensors, achieving excellent computational scalability, and also demonstrate
its usefulness in leveraging side-information provided in form of
mode-network(s).Comment: UAI (Uncertainty in Artificial Intelligence) 201
- …