Statistical Mechanics and Information-Theoretic Perspectives on Complexity in the Earth System

Peer reviewedPublisher PD

Successful network inference from time-series data using Mutual Information Rate

This work uses an information-based methodology to infer the connectivity of complex systems from observed time-series data. We first derive analytically an expression for the Mutual Information Rate (MIR), namely, the amount of information exchanged per unit of time, that can be used to estimate the MIR between two finite-length low-resolution noisy time-series, and then apply it after a proper normalization for the identification of the connectivity structure of small networks of interacting dynamical systems. In particular, we show that our methodology successfully infers the connectivity for heterogeneous networks, different time-series lengths or coupling strengths, and even in the presence of additive noise. Finally, we show that our methodology based on MIR successfully infers the connectivity of networks composed of nodes with different time-scale dynamics, where inference based on Mutual Information fails

Model-free inference of direct network interactions from nonlinear collective dynamics

The topology of interactions in network dynamical systems fundamentally underlies their function. Accelerating technological progress creates massively available data about collective nonlinear dynamics in physical, biological, and technological systems. Detecting direct interaction patterns from those dynamics still constitutes a major open problem. In particular, current nonlinear dynamics approaches mostly require to know a priori a model of the (often high dimensional) system dynamics. Here we develop a model-independent framework for inferring direct interactions solely from recording the nonlinear collective dynamics generated. Introducing an explicit dependency matrix in combination with a block-orthogonal regression algorithm, the approach works reliably across many dynamical regimes, including transient dynamics toward steady states, periodic and non-periodic dynamics, and chaos. Together with its capabilities to reveal network (two point) as well as hypernetwork (e.g., three point) interactions, this framework may thus open up nonlinear dynamics options of inferring direct interaction patterns across systems where no model is known.Comment: 10 pages, 7 figure

Trends in recurrence analysis of dynamical systems

The last decade has witnessed a number of important and exciting developments that had been achieved for improving recurrence plot-based data analysis and to widen its application potential. We will give a brief overview about important and innovative developments, such as computational improvements, alternative recurrence definitions (event-like, multiscale, heterogeneous, and spatio-temporal recurrences) and ideas for parameter selection, theoretical considerations of recurrence quantification measures, new recurrence quantifiers (e.g. for transition detection and causality detection), and correction schemes. New perspectives have recently been opened by combining recurrence plots with machine learning. We finally show open questions and perspectives for futures directions of methodical research

Recurrence flow measure of nonlinear dependence

Couplings in complex real-world systems are often nonlinear and scale dependent. In many cases, it is crucial to consider a multitude of interlinked variables and the strengths of their correlations to adequately fathom the dynamics of a high-dimensional nonlinear system. We propose a recurrence-based dependence measure that quantifies the relationship between multiple time series based on the predictability of their joint evolution. The statistical analysis of recurrence plots (RPs) is a powerful framework in nonlinear time series analysis that has proven to be effective in addressing many fundamental problems, e.g., regime shift detection and identification of couplings. The recurrence flow through an RP exploits artifacts in the formation of diagonal lines, a structure in RPs that reflects periods of predictable dynamics. Using time-delayed variables of a deterministic uni-/multivariate system, lagged dependencies with potentially many time scales can be captured by the recurrence flow measure. Given an RP, no parameters are required for its computation. We showcase the scope of the method for quantifying lagged nonlinear correlations and put a focus on the delay selection problem in time-delay embedding which is often used for attractor reconstruction. The recurrence flow measure of dependence helps to identify non-uniform delays and appears as a promising foundation for a recurrence-based state space reconstruction algorithm

On the potential of time delay neural networks to detect indirect coupling between time series

Determining the coupling between systems remains a topic of active research in the field of complex science. Identifying the proper causal influences in time series can already be very challenging in the trivariate case, particularly when the interactions are non-linear. In this paper, the coupling between three Lorenz systems is investigated with the help of specifically designed artificial neural networks, called time delay neural networks (TDNNs). TDNNs can learn from their previous inputs and are therefore well suited to extract the causal relationship between time series. The performances of the TDNNs tested have always been very positive, showing an excellent capability to identify the correct causal relationships in absence of significant noise. The first tests on the time localization of the mutual influences and the effects of Gaussian noise have also provided very encouraging results. Even if further assessments are necessary, the networks of the proposed architecture have the potential to be a good complement to the other techniques available in the market for the investigation of mutual influences between time serie

NetCausality: A time-delayed neural network tool for causality detection and analysis

The analysis of causality between systems is still an important research activity, which finds application in several fields of science. The software presented is a new tool for causality detection and analysis between time series. The proposed technique is based on time-delayed neural networks (TDNN). The tool is developed in MATLAB and it comprises three main functions. The first one returns the total causality between two or more systems of equations. The second tool is used to find the ‘‘time horizon’’, id est the time delay at which the influence between the systems occurs. The last function is a causality feature detection to determine the time intervals, in which the mutual coupling is sufficiently strong to have a real influence on the target

Unified functional network and nonlinear time series analysis for complex systems science: The pyunicorn package

We introduce the \texttt{pyunicorn} (Pythonic unified complex network and recurrence analysis toolbox) open source software package for applying and combining modern methods of data analysis and modeling from complex network theory and nonlinear time series analysis. \texttt{pyunicorn} is a fully object-oriented and easily parallelizable package written in the language Python. It allows for the construction of functional networks such as climate networks in climatology or functional brain networks in neuroscience representing the structure of statistical interrelationships in large data sets of time series and, subsequently, investigating this structure using advanced methods of complex network theory such as measures and models for spatial networks, networks of interacting networks, node-weighted statistics or network surrogates. Additionally, \texttt{pyunicorn} provides insights into the nonlinear dynamics of complex systems as recorded in uni- and multivariate time series from a non-traditional perspective by means of recurrence quantification analysis (RQA), recurrence networks, visibility graphs and construction of surrogate time series. The range of possible applications of the library is outlined, drawing on several examples mainly from the field of climatology.Comment: 28 pages, 17 figure

Network inference combining mutual information rate and statistical tests

In this paper, we present a method that combines information-theoretical and statistical approaches to infer connec- tivity in complex networks using time-series data. The method is based on estimations of the Mutual Information Rate for pairs of time-series and on statistical significance tests for connectivity acceptance using the false discovery rate method for multiple hypothesis testing. We provide the mathematical background on Mutual Information Rate, discuss the statistical significance tests and the false discovery rate. Further on, we present results for corre- lated normal-variates data, coupled circle and coupled logistic maps, coupled Lorenz systems and coupled stochastic Kuramoto phase oscillators. Following up, we study the effect of noise on the presented methodology in networks of coupled stochastic Kuramoto phase oscillators and of coupling heterogeneity degree on networks of coupled circle maps. We show that the method can infer the correct number and pairs of connected nodes, by means of receiver operating characteristic curves. In the more realistic case of stochastic data, we demonstrate its ability to infer the structure of the initial connectivity matrices. The method is also shown to recover the initial connectivity matrices for dynamics on the nodes of Erd ̋os-R ́enyi and small-world networks with varying coupling heterogeneity in their connections. The highlight of the proposed methodology is its ability to infer the underlying network connectivity based solely on the recorded datasets