1,368 research outputs found
Learning in Markov Random Fields with Contrastive Free Energies
Learning Markov random field (MRF) models is notoriously hard due to the presence of a global normalization factor. In this paper we present a new framework for learning MRF models based on the contrastive free energy (CF) objective function. In this scheme the parameters are updated in an attempt to match the average statistics of the data distribution and a distribution which is (partially or approximately) "relaxed" to the equilibrium distribution. We show that maximum likelihood, mean field, contrastive divergence and pseudo-likelihood objectives can be understood in this paradigm. Moreover, we propose and study a new learning algorithm: the "kstep Kikuchi/Bethe approximation". This algorithm is then tested on a conditional random field model with "skip-chain" edges to model long range interactions in text data. It is demonstrated that with no loss in accuracy, the training time is brought down on average from 19 hours (BP based learning) to 83 minutes, an order of magnitude improvement
Bayesian inference for queueing networks and modeling of internet services
Modern Internet services, such as those at Google, Yahoo!, and Amazon, handle
billions of requests per day on clusters of thousands of computers. Because
these services operate under strict performance requirements, a statistical
understanding of their performance is of great practical interest. Such
services are modeled by networks of queues, where each queue models one of the
computers in the system. A key challenge is that the data are incomplete,
because recording detailed information about every request to a heavily used
system can require unacceptable overhead. In this paper we develop a Bayesian
perspective on queueing models in which the arrival and departure times that
are not observed are treated as latent variables. Underlying this viewpoint is
the observation that a queueing model defines a deterministic transformation
between the data and a set of independent variables called the service times.
With this viewpoint in hand, we sample from the posterior distribution over
missing data and model parameters using Markov chain Monte Carlo. We evaluate
our framework on data from a benchmark Web application. We also present a
simple technique for selection among nested queueing models. We are unaware of
any previous work that considers inference in networks of queues in the
presence of missing data.Comment: Published in at http://dx.doi.org/10.1214/10-AOAS392 the Annals of
Applied Statistics (http://www.imstat.org/aoas/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Probabilistic Inference in Queueing Networks
Although queueing models have long been used to model the performance of computer systems, they are out of favor with practitioners, because they have a reputation for requiring unrealistic distributional assumptions. In fact, these distributional assumptions are used mainly to facilitate analytic approximations such as asymptotics and large-deviations bounds. In this paper, we analyze queueing networks from the probabilistic modeling perspective, applying inference methods from graphical models that afford significantly more modeling flexibility. In particular, we present a Gibbs sampler and stochastic EM algorithm for networks of M/M/1 FIFO queues. As an application of this technique, we localize performance problems in distributed systems from incomplete system trace data. On both synthetic networks and an actual distributed Web application, the model accurately recovers the system’s service time using 1 % of the available trace data.
Inference and Learning in Networks of Queues
Probabilistic models of the performance of computer systems are useful both for predicting system performance in new conditions, and for diagnosing past performance problems. The most popular performance models are networks of queues. However, no current methods exist for parameter estimation or inference in networks of queues with missing data. In this paper, we present a novel viewpoint that combines queueing networks and graphical models, allowing Markov chain Monte Carlo to be applied. We demonstrate the effectiveness of our sampler on real-world data from a benchmark Web application.
Improving Academic Performance Through the Enhancement of Teacher/Student Relationships: The Relationship Teaching Model
The authors present their case for the development of strong and appropriate relationships with students as a key for success in college teaching. The model of Relationship Teaching includes a wide and varied agenda of techniques and commitments with which to strengthen the interpersonal relationships present in the educational environment
Interleaved Factorial Non-Homogeneous Hidden Markov Models for Energy Disaggregation
To reduce energy demand in households it is useful to know which electrical
appliances are in use at what times. Monitoring individual appliances is costly
and intrusive, whereas data on overall household electricity use is more easily
obtained. In this paper, we consider the energy disaggregation problem where a
household's electricity consumption is disaggregated into the component
appliances. The factorial hidden Markov model (FHMM) is a natural model to fit
this data. We enhance this generic model by introducing two constraints on the
state sequence of the FHMM. The first is to use a non-homogeneous Markov chain,
modelling how appliance usage varies over the day, and the other is to enforce
that at most one chain changes state at each time step. This yields a new model
which we call the interleaved factorial non-homogeneous hidden Markov model
(IFNHMM). We evaluated the ability of this model to perform disaggregation in
an ultra-low frequency setting, over a data set of 251 English households. In
this new setting, the IFNHMM outperforms the FHMM in terms of recovering the
energy used by the component appliances, due to that stronger constraints have
been imposed on the states of the hidden Markov chains. Interestingly, we find
that the variability in model performance across households is significant,
underscoring the importance of using larger scale data in the disaggregation
problem.Comment: 5 pages, 1 figure, conference, The NIPS workshop on Machine Learning
for Sustainability, Lake Tahoe, NV, USA, 201
- …