61 research outputs found
Causal Discovery from Temporal Data: An Overview and New Perspectives
Temporal data, representing chronological observations of complex systems,
has always been a typical data structure that can be widely generated by many
domains, such as industry, medicine and finance. Analyzing this type of data is
extremely valuable for various applications. Thus, different temporal data
analysis tasks, eg, classification, clustering and prediction, have been
proposed in the past decades. Among them, causal discovery, learning the causal
relations from temporal data, is considered an interesting yet critical task
and has attracted much research attention. Existing casual discovery works can
be divided into two highly correlated categories according to whether the
temporal data is calibrated, ie, multivariate time series casual discovery, and
event sequence casual discovery. However, most previous surveys are only
focused on the time series casual discovery and ignore the second category. In
this paper, we specify the correlation between the two categories and provide a
systematical overview of existing solutions. Furthermore, we provide public
datasets, evaluation metrics and new perspectives for temporal data casual
discovery.Comment: 52 pages, 6 figure
Block Neural Autoregressive Flow
Normalising flows (NFS) map two density functions via a differentiable
bijection whose Jacobian determinant can be computed efficiently. Recently, as
an alternative to hand-crafted bijections, Huang et al. (2018) proposed neural
autoregressive flow (NAF) which is a universal approximator for density
functions. Their flow is a neural network (NN) whose parameters are predicted
by another NN. The latter grows quadratically with the size of the former and
thus an efficient technique for parametrization is needed. We propose block
neural autoregressive flow (B-NAF), a much more compact universal approximator
of density functions, where we model a bijection directly using a single
feed-forward network. Invertibility is ensured by carefully designing each
affine transformation with block matrices that make the flow autoregressive and
(strictly) monotone. We compare B-NAF to NAF and other established flows on
density estimation and approximate inference for latent variable models. Our
proposed flow is competitive across datasets while using orders of magnitude
fewer parameters.Comment: 12 pages, 3 figures, 3 table
On the Rationality of Explanations in Classification Algorithms
This paper is a first step towards studying the rationality of explanations produced by up-to-date AI systems. Based on the thesis that designing rational explanations for accomplishing trustworthy AI is fundamental for ethics in AI, we study the rationality criteria that explanations in classification algorithms have to meet. In this way, we identify, define, and exemplify characteristic criteria of rational explanations in classification algorithms
Estimating Model Uncertainty of Neural Networks in Sparse Information Form
We present a sparse representation of model uncertainty for Deep Neural Networks (DNNs) where the parameter posterior is approximated with an inverse formulation of the Multivariate Normal Distribution (MND), also known as the information form. The key insight of our work is that the information matrix, i.e. the inverse of the covariance matrix tends to be sparse in its spectrum. Therefore, dimensionality reduction techniques such as low rank approximations (LRA) can be effectively exploited. To achieve this, we develop a novel sparsification algorithm and derive a cost-effective analytical sampler. As a result, we show that the information form can be scalably applied to represent model uncertainty in DNNs. Our exhaustive theoretical analysis and empirical evaluations on various benchmarks show the competitiveness of our approach over the current methods
Model Selection for Bayesian Autoencoders
We develop a novel method for carrying out model selection for Bayesian
autoencoders (BAEs) by means of prior hyper-parameter optimization. Inspired by
the common practice of type-II maximum likelihood optimization and its
equivalence to Kullback-Leibler divergence minimization, we propose to optimize
the distributional sliced-Wasserstein distance (DSWD) between the output of the
autoencoder and the empirical data distribution. The advantages of this
formulation are that we can estimate the DSWD based on samples and handle
high-dimensional problems. We carry out posterior estimation of the BAE
parameters via stochastic gradient Hamiltonian Monte Carlo and turn our BAE
into a generative model by fitting a flexible Dirichlet mixture model in the
latent space. Consequently, we obtain a powerful alternative to variational
autoencoders, which are the preferred choice in modern applications of
autoencoders for representation learning with uncertainty. We evaluate our
approach qualitatively and quantitatively using a vast experimental campaign on
a number of unsupervised learning tasks and show that, in small-data regimes
where priors matter, our approach provides state-of-the-art results,
outperforming multiple competitive baselines
Causality-based Neural Network Repair
Neural networks have had discernible achievements in a wide range of
applications. The wide-spread adoption also raises the concern of their
dependability and reliability. Similar to traditional decision-making programs,
neural networks can have defects that need to be repaired. The defects may
cause unsafe behaviors, raise security concerns or unjust societal impacts. In
this work, we address the problem of repairing a neural network for desirable
properties such as fairness and the absence of backdoor. The goal is to
construct a neural network that satisfies the property by (minimally) adjusting
the given neural network's parameters (i.e., weights). Specifically, we propose
CARE (\textbf{CA}usality-based \textbf{RE}pair), a causality-based neural
network repair technique that 1) performs causality-based fault localization to
identify the `guilty' neurons and 2) optimizes the parameters of the identified
neurons to reduce the misbehavior. We have empirically evaluated CARE on
various tasks such as backdoor removal, neural network repair for fairness and
safety properties. Our experiment results show that CARE is able to repair all
neural networks efficiently and effectively. For fairness repair tasks, CARE
successfully improves fairness by on average. For backdoor removal
tasks, CARE reduces the attack success rate from over to less than
. For safety property repair tasks, CARE reduces the property violation
rate to less than . Results also show that thanks to the causality-based
fault localization, CARE's repair focuses on the misbehavior and preserves the
accuracy of the neural networks
Learning Likelihoods with Conditional Normalizing Flows
Normalizing Flows (NFs) are able to model complicated distributions p(y) with
strong inter-dimensional correlations and high multimodality by transforming a
simple base density p(z) through an invertible neural network under the change
of variables formula. Such behavior is desirable in multivariate structured
prediction tasks, where handcrafted per-pixel loss-based methods inadequately
capture strong correlations between output dimensions. We present a study of
conditional normalizing flows (CNFs), a class of NFs where the base density to
output space mapping is conditioned on an input x, to model conditional
densities p(y|x). CNFs are efficient in sampling and inference, they can be
trained with a likelihood-based objective, and CNFs, being generative flows, do
not suffer from mode collapse or training instabilities. We provide an
effective method to train continuous CNFs for binary problems and in
particular, we apply these CNFs to super-resolution and vessel segmentation
tasks demonstrating competitive performance on standard benchmark datasets in
terms of likelihood and conventional metrics.Comment: 18 pages, 8 Tables, 9 Figures, Preprin
Demographic Parity Inspector: Fairness Audits via the Explanation Space
Even if deployed with the best intentions, machine learning methods can
perpetuate, amplify or even create social biases. Measures of (un-)fairness
have been proposed as a way to gauge the (non-)discriminatory nature of machine
learning models. However, proxies of protected attributes causing
discriminatory effects remain challenging to address. In this work, we propose
a new algorithmic approach that measures group-wise demographic parity
violations and allows us to inspect the causes of inter-group discrimination.
Our method relies on the novel idea of measuring the dependence of a model on
the protected attribute based on the explanation space, an informative space
that allows for more sensitive audits than the primary space of input data or
prediction distributions, and allowing for the assertion of theoretical
demographic parity auditing guarantees. We provide a mathematical analysis,
synthetic examples, and experimental evaluation of real-world data. We release
an open-source Python package with methods, routines, and tutorials
- …