Identifying Nonlinear 1-Step Causal Influences in Presence of Latent Variables

We propose an approach for learning the causal structure in stochastic dynamical systems with a $1$-step functional dependency in the presence of latent variables. We propose an information-theoretic approach that allows us to recover the causal relations among the observed variables as long as the latent variables evolve without exogenous noise. We further propose an efficient learning method based on linear regression for the special sub-case when the dynamics are restricted to be linear. We validate the performance of our approach via numerical simulations

Detecting and quantifying causal associations in large nonlinear time series datasets

Identifying causal relationships and quantifying their strength from observational time series data are key problems in disciplines dealing with complex dynamical systems such as the Earth system or the human body. Data-driven causal inference in such systems is challenging since datasets are often high dimensional and nonlinear with limited sample sizes. Here, we introduce a novel method that flexibly combines linear or nonlinear conditional independence tests with a causal discovery algorithm to estimate causal networks from large-scale time series datasets. We validate the method on time series of well-understood physical mechanisms in the climate system and the human heart and using large-scale synthetic datasets mimicking the typical properties of real-world data. The experiments demonstrate that our method outperforms state-of-the-art techniques in detection power, which opens up entirely new possibilities to discover and quantify causal networks from time series across a range of research fields

Editorial Comment on the Special Issue of "Information in Dynamical Systems and Complex Systems"

This special issue collects contributions from the participants of the "Information in Dynamical Systems and Complex Systems" workshop, which cover a wide range of important problems and new approaches that lie in the intersection of information theory and dynamical systems. The contributions include theoretical characterization and understanding of the different types of information flow and causality in general stochastic processes, inference and identification of coupling structure and parameters of system dynamics, rigorous coarse-grain modeling of network dynamical systems, and exact statistical testing of fundamental information-theoretic quantities such as the mutual information. The collective efforts reported herein reflect a modern perspective of the intimate connection between dynamical systems and information flow, leading to the promise of better understanding and modeling of natural complex systems and better/optimal design of engineering systems

Measuring information-transfer delays

In complex networks such as gene networks, traffic systems or brain circuits it is important to understand how long it takes for the different parts of the network to effectively influence one another. In the brain, for example, axonal delays between brain areas can amount to several tens of milliseconds, adding an intrinsic component to any timing-based processing of information. Inferring neural interaction delays is thus needed to interpret the information transfer revealed by any analysis of directed interactions across brain structures. However, a robust estimation of interaction delays from neural activity faces several challenges if modeling assumptions on interaction mechanisms are wrong or cannot be made. Here, we propose a robust estimator for neuronal interaction delays rooted in an information-theoretic framework, which allows a model-free exploration of interactions. In particular, we extend transfer entropy to account for delayed source-target interactions, while crucially retaining the conditioning on the embedded target state at the immediately previous time step. We prove that this particular extension is indeed guaranteed to identify interaction delays between two coupled systems and is the only relevant option in keeping with Wiener’s principle of causality. We demonstrate the performance of our approach in detecting interaction delays on finite data by numerical simulations of stochastic and deterministic processes, as well as on local field potential recordings. We also show the ability of the extended transfer entropy to detect the presence of multiple delays, as well as feedback loops. While evaluated on neuroscience data, we expect the estimator to be useful in other fields dealing with network dynamics

Brain networks under attack : robustness properties and the impact of lesions

A growing number of studies approach the brain as a complex network, the so-called ‘connectome’. Adopting this framework, we examine what types or extent of damage the brain can withstand—referred to as network ‘robustness’—and conversely, which kind of distortions can be expected after brain lesions. To this end, we review computational lesion studies and empirical studies investigating network alterations in brain tumour, stroke and traumatic brain injury patients. Common to these three types of focal injury is that there is no unequivocal relationship between the anatomical lesion site and its topological characteristics within the brain network. Furthermore, large-scale network effects of these focal lesions are compared to those of a widely studied multifocal neurodegenerative disorder, Alzheimer’s disease, in which central parts of the connectome are preferentially affected. Results indicate that human brain networks are remarkably resilient to different types of lesions, compared to other types of complex networks such as random or scale-free networks. However, lesion effects have been found to depend critically on the topological position of the lesion. In particular, damage to network hub regions—and especially those connecting different subnetworks—was found to cause the largest disturbances in network organization. Regardless of lesion location, evidence from empirical and computational lesion studies shows that lesions cause significant alterations in global network topology. The direction of these changes though remains to be elucidated. Encouragingly, both empirical and modelling studies have indicated that after focal damage, the connectome carries the potential to recover at least to some extent, with normalization of graph metrics being related to improved behavioural and cognitive functioning. To conclude, we highlight possible clinical implications of these findings, point out several methodological limitations that pertain to the study of brain diseases adopting a network approach, and provide suggestions for future research

Inferring causation from time series in Earth system sciences

The heart of the scientific enterprise is a rational effort to understand the causes behind the phenomena we observe. In large-scale complex dynamical systems such as the Earth system, real experiments are rarely feasible. However, a rapidly increasing amount of observational and simulated data opens up the use of novel data-driven causal methods beyond the commonly adopted correlation techniques. Here, we give an overview of causal inference frameworks and identify promising generic application cases common in Earth system sciences and beyond. We discuss challenges and initiate the benchmark platform causeme.net to close the gap between method users and developers. © 2019, The Author(s)

Dynamical network stability analysis of multiple biological ages provides a framework for understanding the aging process

Widespread interest in non-destructive biomarkers of aging has led to a curse of plenty: a multitude of biological ages that each proffers a 'true' health-adjusted age of an individual. While each measure provides salient information on the aging process, they are each univariate, in contrast to the "hallmark" and "pillar" theories of aging which are explicitly multidimensional, multicausal and multiscale. Fortunately, multiple biological ages can be systematically combined into a multidimensional network representation. The interaction network between these biological ages permits analysis of the multidimensional effects of aging, as well as quantification of causal influences during both natural aging and, potentially, after anti-aging intervention. The behaviour of the system as a whole can then be explored using dynamical network stability analysis which identifies new, efficient biomarkers that quantify long term resilience scores on the timescale between measurements (years). We demonstrate this approach using a set of 8 biological ages from the longitudinal Swedish Adoption/Twin Study of Aging (SATSA). After extracting an interaction network between these biological ages, we observed that physiological age, a proxy for cardiometabolic health, serves as a central node in the network, implicating it as a key vulnerability for slow, age-related decline. We furthermore show that while the system as a whole is stable, there is a weakly stable direction along which recovery is slow - on the timescale of a human lifespan. This slow direction provides an aging biomarker which correlates strongly with chronological age and predicts longitudinal decline in health - suggesting that it estimates an important driver of age-related changes.Comment: 65 pages including supplementa

Estimating causal networks in biosphere–atmosphere interaction with the PCMCI approach

Local meteorological conditions and biospheric activity are tightly coupled. Understanding these links is an essential prerequisite for predicting the Earth system under climate change conditions. However, many empirical studies on the interaction between the biosphere and the atmosphere are based on correlative approaches that are not able to deduce causal paths, and only very few studies apply causal discovery methods. Here, we use a recently proposed causal graph discovery algorithm, which aims to reconstruct the causal dependency structure underlying a set of time series. We explore the potential of this method to infer temporal dependencies in biosphere-atmosphere interactions. Specifically we address the following questions: How do periodicity and heteroscedasticity influence causal detection rates, i.e. the detection of existing and non-existing links? How consistent are results for noise-contaminated data? Do results exhibit an increased information content that justifies the use of this causal-inference method? We explore the first question using artificial time series with well known dependencies that mimic real-world biosphere-atmosphere interactions. The two remaining questions are addressed jointly in two case studies utilizing observational data. Firstly, we analyse three replicated eddy covariance datasets from a Mediterranean ecosystem at half hourly time resolution allowing us to understand the impact of measurement uncertainties. Secondly, we analyse global NDVI time series (GIMMS 3g) along with gridded climate data to study large-scale climatic drivers of vegetation greenness. Overall, the results confirm the capacity of the causal discovery method to extract time-lagged linear dependencies under realistic settings. The violation of the method's assumptions increases the likelihood to detect false links. Nevertheless, we consistently identify interaction patterns in observational data. Our findings suggest that estimating a directed biosphere-atmosphere network at the ecosystem level can offer novel possibilities to unravel complex multi-directional interactions. Other than classical correlative approaches, our findings are constrained to a few meaningful set of relations which can be powerful insights for the evaluation of terrestrial ecosystem models

Sparse Learning for Variable Selection with Structures and Nonlinearities

In this thesis we discuss machine learning methods performing automated variable selection for learning sparse predictive models. There are multiple reasons for promoting sparsity in the predictive models. By relying on a limited set of input variables the models naturally counteract the overfitting problem ubiquitous in learning from finite sets of training points. Sparse models are cheaper to use for predictions, they usually require lower computational resources and by relying on smaller sets of inputs can possibly reduce costs for data collection and storage. Sparse models can also contribute to better understanding of the investigated phenomenons as they are easier to interpret than full models.Comment: PhD thesi