3,041 research outputs found
Modeling Topic and Role Information in Meetings using the Hierarchical Dirichlet Process
Abstract. In this paper, we address the modeling of topic and role information in multiparty meetings, via a nonparametric Bayesian model called the hierarchical Dirichlet process. This model provides a powerful solution to topic modeling and a flexible framework for the incorporation of other cues such as speaker role information. We present our modeling framework for topic and role on the AMI Meeting Corpus, and illustrate the effectiveness of the approach in the context of adapting a baseline language model in a large-vocabulary automatic speech recognition system for multiparty meetings. The adapted LM produces significant improvements in terms of both perplexity and word error rate.
Detecting Variability in Massive Astronomical Time-Series Data I: application of an infinite Gaussian mixture model
We present a new framework to detect various types of variable objects within
massive astronomical time-series data. Assuming that the dominant population of
objects is non-variable, we find outliers from this population by using a
non-parametric Bayesian clustering algorithm based on an infinite
GaussianMixtureModel (GMM) and the Dirichlet Process. The algorithm extracts
information from a given dataset, which is described by six variability
indices. The GMM uses those variability indices to recover clusters that are
described by six-dimensional multivariate Gaussian distributions, allowing our
approach to consider the sampling pattern of time-series data, systematic
biases, the number of data points for each light curve, and photometric
quality. Using the Northern Sky Variability Survey data, we test our approach
and prove that the infinite GMM is useful at detecting variable objects, while
providing statistical inference estimation that suppresses false detection. The
proposed approach will be effective in the exploration of future surveys such
as GAIA, Pan-Starrs, and LSST, which will produce massive time-series data.Comment: accepted for publication in MNRA
Producing power-law distributions and damping word frequencies with two-stage language models
Standard statistical models of language fail to capture one of the most striking properties of natural languages: the power-law distribution in the frequencies of word tokens. We present a framework for developing statisticalmodels that can generically produce power laws, breaking generativemodels into two stages. The first stage, the generator, can be any standard probabilistic model, while the second stage, the adaptor, transforms the word frequencies of this model to provide a closer match to natural language. We show that two commonly used Bayesian models, the Dirichlet-multinomial model and the Dirichlet process, can be viewed as special cases of our framework. We discuss two stochastic processes-the Chinese restaurant process and its two-parameter generalization based on the Pitman-Yor process-that can be used as adaptors in our framework to produce power-law distributions over word frequencies. We show that these adaptors justify common estimation procedures based on logarithmic or inverse-power transformations of empirical frequencies. In addition, taking the Pitman-Yor Chinese restaurant process as an adaptor justifies the appearance of type frequencies in formal analyses of natural language and improves the performance of a model for unsupervised learning of morphology.48 page(s
The supervised hierarchical Dirichlet process
We propose the supervised hierarchical Dirichlet process (sHDP), a
nonparametric generative model for the joint distribution of a group of
observations and a response variable directly associated with that whole group.
We compare the sHDP with another leading method for regression on grouped data,
the supervised latent Dirichlet allocation (sLDA) model. We evaluate our method
on two real-world classification problems and two real-world regression
problems. Bayesian nonparametric regression models based on the Dirichlet
process, such as the Dirichlet process-generalised linear models (DP-GLM) have
previously been explored; these models allow flexibility in modelling nonlinear
relationships. However, until now, Hierarchical Dirichlet Process (HDP)
mixtures have not seen significant use in supervised problems with grouped data
since a straightforward application of the HDP on the grouped data results in
learnt clusters that are not predictive of the responses. The sHDP solves this
problem by allowing for clusters to be learnt jointly from the group structure
and from the label assigned to each group.Comment: 14 page
- …