10,633 research outputs found
Recommended from our members
Complexity of probabilistic inference in belief nets--an experimental study
Graduation date: 1991There are three families of exact methods used for probabilistic inference in\ud
belief nets. It is necessary to compare them and analyze the advantages and\ud
the disadvantages of each algorithm, and know the time cost of making\ud
inferences in a given belief network. This paper discusses the factors that\ud
influence the computation time of each algorithm, presents the predictive model\ud
of the time complexity for each algorithm and shows the statistical results of\ud
testing the algorithms with randomly generated belief networks
Incremental Probabilistic Inference
Propositional representation services such as truth maintenance systems offer
powerful support for incremental, interleaved, problem-model construction and
evaluation. Probabilistic inference systems, in contrast, have lagged behind in
supporting this incrementality typically demanded by problem solvers. The
problem, we argue, is that the basic task of probabilistic inference is
typically formulated at too large a grain-size. We show how a system built
around a smaller grain-size inference task can have the desired incrementality
and serve as the basis for a low-level (propositional) probabilistic
representation service.Comment: Appears in Proceedings of the Ninth Conference on Uncertainty in
Artificial Intelligence (UAI1993
Is 'Unsupervised Learning' a Misconceived Term?
Is all of machine learning supervised to some degree? The field of machine
learning has traditionally been categorized pedagogically into
; where supervised learning has typically
referred to learning from labeled data, while unsupervised learning has
typically referred to learning from unlabeled data. In this paper, we assert
that all machine learning is in fact supervised to some degree, and that the
scope of supervision is necessarily commensurate to the scope of learning
potential. In particular, we argue that clustering algorithms such as k-means,
and dimensionality reduction algorithms such as principal component analysis,
variational autoencoders, and deep belief networks are each internally
supervised by the data itself to learn their respective representations of its
features. Furthermore, these algorithms are not capable of external inference
until their respective outputs (clusters, principal components, or
representation codes) have been identified and externally labeled in effect. As
such, they do not suffice as examples of unsupervised learning. We propose that
the categorization `supervised vs unsupervised learning' be dispensed with, and
instead, learning algorithms be categorized as either
(or both). We believe this change in
perspective will yield new fundamental insights into the structure and
character of data and of learning algorithms.Comment: 9 pages, 3 figure
Automating Computer Bottleneck Detection with Belief Nets
We describe an application of belief networks to the diagnosis of bottlenecks
in computer systems. The technique relies on a high-level functional model of
the interaction between application workloads, the Windows NT operating system,
and system hardware. Given a workload description, the model predicts the
values of observable system counters available from the Windows NT performance
monitoring tool. Uncertainty in workloads, predictions, and counter values are
characterized with Gaussian distributions. During diagnostic inference, we use
observed performance monitor values to find the most probable assignment to the
workload parameters. In this paper we provide some background on automated
bottleneck detection, describe the structure of the system model, and discuss
empirical procedures for model calibration and verification. Part of the
calibration process includes generating a dataset to estimate a multivariate
Gaussian error model. Initial results in diagnosing bottlenecks are presented.Comment: Appears in Proceedings of the Eleventh Conference on Uncertainty in
Artificial Intelligence (UAI1995
Learning Class-Level Bayes Nets for Relational Data
Many databases store data in relational format, with different types of
entities and information about links between the entities. The field of
statistical-relational learning (SRL) has developed a number of new statistical
models for such data. In this paper we focus on learning class-level or
first-order dependencies, which model the general database statistics over
attributes of linked objects and links (e.g., the percentage of A grades given
in computer science classes). Class-level statistical relationships are
important in themselves, and they support applications like policy making,
strategic planning, and query optimization. Most current SRL methods find
class-level dependencies, but their main task is to support instance-level
predictions about the attributes or links of specific entities. We focus only
on class-level prediction, and describe algorithms for learning class-level
models that are orders of magnitude faster for this task. Our algorithms learn
Bayes nets with relational structure, leveraging the efficiency of single-table
nonrelational Bayes net learners. An evaluation of our methods on three data
sets shows that they are computationally feasible for realistic table sizes,
and that the learned structures represent the statistical information in the
databases well. After learning compiles the database statistics into a Bayes
net, querying these statistics via Bayes net inference is faster than with SQL
queries, and does not depend on the size of the database.Comment: 14 pages (2 column
Real-Time Inference with Large-Scale Temporal Bayes Nets
An increasing number of applications require real-time reasoning under
uncertainty with streaming input. The temporal (dynamic) Bayes net formalism
provides a powerful representational framework for such applications. However,
existing exact inference algorithms for dynamic Bayes nets do not scale to the
size of models required for real world applications which often contain
hundreds or even thousands of variables for each time slice. In addition,
existing algorithms were not developed with real-time processing in mind. We
have developed a new computational approach to support real-time exact
inference in large temporal Bayes nets. Our approach tackles scalability by
recognizing that the complexity of the inference depends on the number of
interface nodes between time slices and by exploiting the distinction between
static and dynamic nodes in order to reduce the number of interface nodes and
to factorize their joint probability distribution. We approach the real-time
issue by organizing temporal Bayes nets into static representations, and then
using the symbolic probabilistic inference algorithm to derive analytic
expressions for the static representations. The parts of these expressions that
do not change at each time step are pre-computed. The remaining parts are
compiled into efficient procedural code so that the memory and CPU resources
required by the inference are small and fixed.Comment: Appears in Proceedings of the Eighteenth Conference on Uncertainty in
Artificial Intelligence (UAI2002
A General Algorithm for Approximate Inference and its Application to Hybrid Bayes Nets
The clique tree algorithm is the standard method for doing inference in
Bayesian networks. It works by manipulating clique potentials - distributions
over the variables in a clique. While this approach works well for many
networks, it is limited by the need to maintain an exact representation of the
clique potentials. This paper presents a new unified approach that combines
approximate inference and the clique tree algorithm, thereby circumventing this
limitation. Many known approximate inference algorithms can be viewed as
instances of this approach. The algorithm essentially does clique tree
propagation, using approximate inference to estimate the densities in each
clique. In many settings, the computation of the approximate clique potential
can be done easily using statistical importance sampling. Iterations are used
to gradually improve the quality of the estimation.Comment: Appears in Proceedings of the Fifteenth Conference on Uncertainty in
Artificial Intelligence (UAI1999
Deep Regression Bayesian Network and Its Applications
Deep directed generative models have attracted much attention recently due to
their generative modeling nature and powerful data representation ability. In
this paper, we review different structures of deep directed generative models
and the learning and inference algorithms associated with the structures. We
focus on a specific structure that consists of layers of Bayesian Networks due
to the property of capturing inherent and rich dependencies among latent
variables. The major difficulty of learning and inference with deep directed
models with many latent variables is the intractable inference due to the
dependencies among the latent variables and the exponential number of latent
variable configurations. Current solutions use variational methods often
through an auxiliary network to approximate the posterior probability
inference. In contrast, inference can also be performed directly without using
any auxiliary network to maximally preserve the dependencies among the latent
variables. Specifically, by exploiting the sparse representation with the
latent space, max-max instead of max-sum operation can be used to overcome the
exponential number of latent configurations. Furthermore, the max-max operation
and augmented coordinate ascent are applied to both supervised and unsupervised
learning as well as to various inference. Quantitative evaluations on benchmark
datasets of different models are given for both data representation and feature
learning tasks.Comment: Accepted to IEEE Signal Processing Magazin
Loglinear models for first-order probabilistic reasoning
Recent work on loglinear models in probabilistic constraint logic programming
is applied to first-order probabilistic reasoning. Probabilities are defined
directly on the proofs of atomic formulae, and by marginalisation on the atomic
formulae themselves. We use Stochastic Logic Programs (SLPs) composed of
labelled and unlabelled definite clauses to define the proof probabilities. We
have a conservative extension of first-order reasoning, so that, for example,
there is a one-one mapping between logical and random variables. We show how,
in this framework, Inductive Logic Programming (ILP) can be used to induce the
features of a loglinear model from data. We also compare the presented
framework with other approaches to first-order probabilistic reasoning.Comment: Appears in Proceedings of the Fifteenth Conference on Uncertainty in
Artificial Intelligence (UAI1999
Sample-and-Accumulate Algorithms for Belief Updating in Bayes Networks
Belief updating in Bayes nets, a well known computationally hard problem, has
recently been approximated by several deterministic algorithms, and by various
randomized approximation algorithms. Deterministic algorithms usually provide
probability bounds, but have an exponential runtime. Some randomized schemes
have a polynomial runtime, but provide only probability estimates. We present
randomized algorithms that enumerate high-probability partial instantiations,
resulting in probability bounds. Some of these algorithms are also sampling
algorithms. Specifically, we introduce and evaluate a variant of backward
sampling, both as a sampling algorithm and as a randomized enumeration
algorithm. We also relax the implicit assumption used by both sampling and
accumulation algorithms, that query nodes must be instantiated in all the
samples.Comment: Appears in Proceedings of the Twelfth Conference on Uncertainty in
Artificial Intelligence (UAI1996
- …