49 research outputs found
Learning Credal Sum-Product Networks
Probabilistic representations, such as Bayesian and Markov networks, are
fundamental to much of statistical machine learning. Thus, learning
probabilistic representations directly from data is a deep challenge, the main
computational bottleneck being inference that is intractable. Tractable
learning is a powerful new paradigm that attempts to learn distributions that
support efficient probabilistic querying. By leveraging local structure,
representations such as sum-product networks (SPNs) can capture high tree-width
models with many hidden layers, essentially a deep architecture, while still
admitting a range of probabilistic queries to be computable in time polynomial
in the network size. While the progress is impressive, numerous data sources
are incomplete, and in the presence of missing data, structure learning methods
nonetheless revert to single distributions without characterizing the loss in
confidence. In recent work, credal sum-product networks, an imprecise extension
of sum-product networks, were proposed to capture this robustness angle. In
this work, we are interested in how such representations can be learnt and thus
study how the computational machinery underlying tractable learning and
inference can be generalized for imprecise probabilities.Comment: Accepted to AKBC 202
Zero-shot Task Preference Addressing Enabled by Imprecise Bayesian Continual Learning
Like generic multi-task learning, continual learning has the nature of
multi-objective optimization, and therefore faces a trade-off between the
performance of different tasks. That is, to optimize for the current task
distribution, it may need to compromise performance on some tasks to improve on
others. This means there exist multiple models that are each optimal at
different times, each addressing a distinct task-performance trade-off.
Researchers have discussed how to train particular models to address specific
preferences on these trade-offs. However, existing algorithms require
additional sample overheads -- a large burden when there are multiple, possibly
infinitely many, preferences. As a response, we propose Imprecise Bayesian
Continual Learning (IBCL). Upon a new task, IBCL (1) updates a knowledge base
in the form of a convex hull of model parameter distributions and (2) obtains
particular models to address preferences with zero-shot. That is, IBCL does not
require any additional training overhead to construct preference-addressing
models from its knowledge base. We show that models obtained by IBCL have
guarantees in identifying the preferred parameters. Moreover, experiments show
that IBCL is able to locate the Pareto set of parameters given a preference,
maintain similar to better performance than baseline methods, and significantly
reduce training overhead via zero-shot preference addressing
Generalized belief change with imprecise probabilities and graphical models
We provide a theoretical investigation of probabilistic belief revision in complex frameworks, under extended conditions of uncertainty, inconsistency and imprecision. We motivate our kinematical approach by specializing our discussion to probabilistic reasoning with graphical models, whose modular representation allows for efficient inference. Most results in this direction are derived from the relevant work of Chan and Darwiche (2005), that first proved the inter-reducibility of virtual and probabilistic evidence. Such forms of information, deeply distinct in their meaning, are extended to the conditional and imprecise frameworks, allowing further generalizations, e.g. to experts' qualitative assessments. Belief aggregation and iterated revision of a rational agent's belief are also explored
Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods
The notion of uncertainty is of major importance in machine learning and
constitutes a key element of machine learning methodology. In line with the
statistical tradition, uncertainty has long been perceived as almost synonymous
with standard probability and probabilistic predictions. Yet, due to the
steadily increasing relevance of machine learning for practical applications
and related issues such as safety requirements, new problems and challenges
have recently been identified by machine learning scholars, and these problems
may call for new methodological developments. In particular, this includes the
importance of distinguishing between (at least) two different types of
uncertainty, often referred to as aleatoric and epistemic. In this paper, we
provide an introduction to the topic of uncertainty in machine learning as well
as an overview of attempts so far at handling uncertainty in general and
formalizing this distinction in particular.Comment: 59 page
Argumentation as a practical foundation for decision theory
Imperial Users onl