225 research outputs found
Forecasting and Granger Modelling with Non-linear Dynamical Dependencies
Traditional linear methods for forecasting multivariate time series are not
able to satisfactorily model the non-linear dependencies that may exist in
non-Gaussian series. We build on the theory of learning vector-valued functions
in the reproducing kernel Hilbert space and develop a method for learning
prediction functions that accommodate such non-linearities. The method not only
learns the predictive function but also the matrix-valued kernel underlying the
function search space directly from the data. Our approach is based on learning
multiple matrix-valued kernels, each of those composed of a set of input
kernels and a set of output kernels learned in the cone of positive
semi-definite matrices. In addition to superior predictive performance in the
presence of strong non-linearities, our method also recovers the hidden dynamic
relationships between the series and thus is a new alternative to existing
graphical Granger techniques.Comment: Accepted for ECML-PKDD 201
Sparse Learning for Variable Selection with Structures and Nonlinearities
In this thesis we discuss machine learning methods performing automated
variable selection for learning sparse predictive models. There are multiple
reasons for promoting sparsity in the predictive models. By relying on a
limited set of input variables the models naturally counteract the overfitting
problem ubiquitous in learning from finite sets of training points. Sparse
models are cheaper to use for predictions, they usually require lower
computational resources and by relying on smaller sets of inputs can possibly
reduce costs for data collection and storage. Sparse models can also contribute
to better understanding of the investigated phenomenons as they are easier to
interpret than full models.Comment: PhD thesi
Causal Discovery from Temporal Data: An Overview and New Perspectives
Temporal data, representing chronological observations of complex systems,
has always been a typical data structure that can be widely generated by many
domains, such as industry, medicine and finance. Analyzing this type of data is
extremely valuable for various applications. Thus, different temporal data
analysis tasks, eg, classification, clustering and prediction, have been
proposed in the past decades. Among them, causal discovery, learning the causal
relations from temporal data, is considered an interesting yet critical task
and has attracted much research attention. Existing casual discovery works can
be divided into two highly correlated categories according to whether the
temporal data is calibrated, ie, multivariate time series casual discovery, and
event sequence casual discovery. However, most previous surveys are only
focused on the time series casual discovery and ignore the second category. In
this paper, we specify the correlation between the two categories and provide a
systematical overview of existing solutions. Furthermore, we provide public
datasets, evaluation metrics and new perspectives for temporal data casual
discovery.Comment: 52 pages, 6 figure
Kernel-based independence tests for causal structure learning on functional data
Measurements of systems taken along a continuous functional dimension, such
as time or space, are ubiquitous in many fields, from the physical and
biological sciences to economics and engineering.Such measurements can be
viewed as realisations of an underlying smooth process sampled over the
continuum. However, traditional methods for independence testing and causal
learning are not directly applicable to such data, as they do not take into
account the dependence along the functional dimension. By using specifically
designed kernels, we introduce statistical tests for bivariate, joint, and
conditional independence for functional variables. Our method not only extends
the applicability to functional data of the HSIC and its d-variate version
(d-HSIC), but also allows us to introduce a test for conditional independence
by defining a novel statistic for the CPT based on the HSCIC, with optimised
regularisation strength estimated through an evaluation rejection rate. Our
empirical results of the size and power of these tests on synthetic functional
data show good performance, and we then exemplify their application to several
constraint- and regression-based causal structure learning problems, including
both synthetic examples and real socio-economic data
D'ya like DAGs? A Survey on Structure Learning and Causal Discovery
Causal reasoning is a crucial part of science and human intelligence. In
order to discover causal relationships from data, we need structure discovery
methods. We provide a review of background theory and a survey of methods for
structure discovery. We primarily focus on modern, continuous optimization
methods, and provide reference to further resources such as benchmark datasets
and software packages. Finally, we discuss the assumptive leap required to take
us from structure to causality.Comment: 35 page
Online Learning with Multiple Operator-valued Kernels
We consider the problem of learning a vector-valued function f in an online
learning setting. The function f is assumed to lie in a reproducing Hilbert
space of operator-valued kernels. We describe two online algorithms for
learning f while taking into account the output structure. A first contribution
is an algorithm, ONORMA, that extends the standard kernel-based online learning
algorithm NORMA from scalar-valued to operator-valued setting. We report a
cumulative error bound that holds both for classification and regression. We
then define a second algorithm, MONORMA, which addresses the limitation of
pre-defining the output structure in ONORMA by learning sequentially a linear
combination of operator-valued kernels. Our experiments show that the proposed
algorithms achieve good performance results with low computational cost
Kernel-based independence tests for causal structure learning on functional data
Measurements of systems taken along a continuous functional dimension, such as time or space, are ubiquitous in many fields, from the physical and biological sciences to economics and engineering. Such measurements can be viewed as realisations of an underlying smooth process sampled over the continuum. However, traditional methods for independence testing and causal learning are not directly applicable to such data, as they do not take into account the dependence along the functional dimension. By using specifically designed kernels, we introduce statistical tests for bivariate, joint, and conditional independence for functional variables. Our method not only extends the applicability to functional data of the Hilbert–Schmidt independence criterion (hsic) and its d-variate version (d-hsic), but also allows us to introduce a test for conditional independence by defining a novel statistic for the conditional permutation test (cpt) based on the Hilbert–Schmidt conditional independence criterion (hscic), with optimised regularisation strength estimated through an evaluation rejection rate. Our empirical results of the size and power of these tests on synthetic functional data show good performance, and we then exemplify their application to several constraint- and regression-based causal structure learning problems, including both synthetic examples and real socioeconomic data
- …