1,717 research outputs found
Computing Upper and Lower Bounds on Likelihoods in Intractable Networks
We present techniques for computing upper and lower bounds on the likelihoods of partial instantiations of variables in sigmoid and noisy-OR networks. The bounds determine confidence intervals for the desired likelihoods and become useful when the size of the network (or clique size) precludes exact computations. We illustrate the tightness of the obtained bounds by numerical experiments
Mean Field Theory for Sigmoid Belief Networks
We develop a mean field theory for sigmoid belief networks based on ideas
from statistical mechanics. Our mean field theory provides a tractable
approximation to the true probability distribution in these networks; it also
yields a lower bound on the likelihood of evidence. We demonstrate the utility
of this framework on a benchmark problem in statistical pattern
recognition---the classification of handwritten digits.Comment: See http://www.jair.org/ for any accompanying file
Discrete Temporal Models of Social Networks
We propose a family of statistical models for social network evolution over
time, which represents an extension of Exponential Random Graph Models (ERGMs).
Many of the methods for ERGMs are readily adapted for these models, including
maximum likelihood estimation algorithms. We discuss models of this type and
their properties, and give examples, as well as a demonstration of their use
for hypothesis testing and classification. We believe our temporal ERG models
represent a useful new framework for modeling time-evolving social networks,
and rewiring networks from other domains such as gene regulation circuitry, and
communication networks
Efficient Optimization of Loops and Limits with Randomized Telescoping Sums
We consider optimization problems in which the objective requires an inner
loop with many steps or is the limit of a sequence of increasingly costly
approximations. Meta-learning, training recurrent neural networks, and
optimization of the solutions to differential equations are all examples of
optimization problems with this character. In such problems, it can be
expensive to compute the objective function value and its gradient, but
truncating the loop or using less accurate approximations can induce biases
that damage the overall solution. We propose randomized telescope (RT) gradient
estimators, which represent the objective as the sum of a telescoping series
and sample linear combinations of terms to provide cheap unbiased gradient
estimates. We identify conditions under which RT estimators achieve
optimization convergence rates independent of the length of the loop or the
required accuracy of the approximation. We also derive a method for tuning RT
estimators online to maximize a lower bound on the expected decrease in loss
per unit of computation. We evaluate our adaptive RT estimators on a range of
applications including meta-optimization of learning rates, variational
inference of ODE parameters, and training an LSTM to model long sequences
- …