91,405 research outputs found
Tractability through Exchangeability: A New Perspective on Efficient Probabilistic Inference
Exchangeability is a central notion in statistics and probability theory. The
assumption that an infinite sequence of data points is exchangeable is at the
core of Bayesian statistics. However, finite exchangeability as a statistical
property that renders probabilistic inference tractable is less
well-understood. We develop a theory of finite exchangeability and its relation
to tractable probabilistic inference. The theory is complementary to that of
independence and conditional independence. We show that tractable inference in
probabilistic models with high treewidth and millions of variables can be
understood using the notion of finite (partial) exchangeability. We also show
that existing lifted inference algorithms implicitly utilize a combination of
conditional independence and partial exchangeability.Comment: In Proceedings of the 28th AAAI Conference on Artificial Intelligenc
Multiple perspectives on the concept of conditional probability
Conditional probability is a key to the subjectivist theory of probability; however, it plays a subsidiary role in the usual conception of probability where its counterpart, namely independence is of basic importance. The paper investigates these concepts from various perspectives in order to shed light on their multi-faceted character. We will include the mathematical, philosophical, and educational perspectives. Furthermore, we will inspect conditional probability from the corners of competing ideas and solving strategies. For the comprehension of conditional probability, a wider approach is urgently needed to overcome the well-known problems in learning the concepts, which seem nearly unaffected by teaching
Semi-Supervised Learning, Causality and the Conditional Cluster Assumption
While the success of semi-supervised learning (SSL) is still not fully
understood, Sch\"olkopf et al. (2012) have established a link to the principle
of independent causal mechanisms. They conclude that SSL should be impossible
when predicting a target variable from its causes, but possible when predicting
it from its effects. Since both these cases are somewhat restrictive, we extend
their work by considering classification using cause and effect features at the
same time, such as predicting disease from both risk factors and symptoms.
While standard SSL exploits information contained in the marginal distribution
of all inputs (to improve the estimate of the conditional distribution of the
target given inputs), we argue that in our more general setting we should use
information in the conditional distribution of effect features given causal
features. We explore how this insight generalises the previous understanding,
and how it relates to and can be exploited algorithmically for SSL.Comment: 36th Conference on Uncertainty in Artificial Intelligence (2020)
(Previously presented at the NeurIPS 2019 workshop "Do the right thing":
machine learning and causal inference for improved decision making,
Vancouver, Canada.
Understanding and Comparing Scalable Gaussian Process Regression for Big Data
As a non-parametric Bayesian model which produces informative predictive
distribution, Gaussian process (GP) has been widely used in various fields,
like regression, classification and optimization. The cubic complexity of
standard GP however leads to poor scalability, which poses challenges in the
era of big data. Hence, various scalable GPs have been developed in the
literature in order to improve the scalability while retaining desirable
prediction accuracy. This paper devotes to investigating the methodological
characteristics and performance of representative global and local scalable GPs
including sparse approximations and local aggregations from four main
perspectives: scalability, capability, controllability and robustness. The
numerical experiments on two toy examples and five real-world datasets with up
to 250K points offer the following findings. In terms of scalability, most of
the scalable GPs own a time complexity that is linear to the training size. In
terms of capability, the sparse approximations capture the long-term spatial
correlations, the local aggregations capture the local patterns but suffer from
over-fitting in some scenarios. In terms of controllability, we could improve
the performance of sparse approximations by simply increasing the inducing
size. But this is not the case for local aggregations. In terms of robustness,
local aggregations are robust to various initializations of hyperparameters due
to the local attention mechanism. Finally, we highlight that the proper hybrid
of global and local scalable GPs may be a promising way to improve both the
model capability and scalability for big data.Comment: 25 pages, 15 figures, preprint submitted to KB
50 Years of Test (Un)fairness: Lessons for Machine Learning
Quantitative definitions of what is unfair and what is fair have been
introduced in multiple disciplines for well over 50 years, including in
education, hiring, and machine learning. We trace how the notion of fairness
has been defined within the testing communities of education and hiring over
the past half century, exploring the cultural and social context in which
different fairness definitions have emerged. In some cases, earlier definitions
of fairness are similar or identical to definitions of fairness in current
machine learning research, and foreshadow current formal work. In other cases,
insights into what fairness means and how to measure it have largely gone
overlooked. We compare past and current notions of fairness along several
dimensions, including the fairness criteria, the focus of the criteria (e.g., a
test, a model, or its use), the relationship of fairness to individuals,
groups, and subgroups, and the mathematical method for measuring fairness
(e.g., classification, regression). This work points the way towards future
research and measurement of (un)fairness that builds from our modern
understanding of fairness while incorporating insights from the past.Comment: FAT* '19: Conference on Fairness, Accountability, and Transparency
(FAT* '19), January 29--31, 2019, Atlanta, GA, US
- …