Testing different methodologies for Granger causality estimation: A simulation study

Granger causality (GC) is a method for determining whether and how two time series exert causal influences one over the other. As it is easy to implement through vector autoregressive (VAR) models and can be generalized to the multivariate case, GC has spread in many different areas of research such as neuroscience and network physiology. In its basic formulation, the computation of GC involves two different regressions, taking respectively into account the whole past history of the investigated multivariate time series (full model) and the past of all time series except the putatively causal time series (restricted model). However, the restricted model cannot be represented through a finite order VAR process and, when few data samples are available or the number of time series is very high, the estimation of GC exhibits a strong reduction in accuracy. To mitigate these problems, improved estimation strategies have been recently implemented, including state space (SS) models and partial conditioning (PC) approaches. In this work, we propose a new method to compute GC which combines SS and PC and tests it together with other four commonly used estimation approaches. In simulated networks of linearly interacting time series, we show the possibility to reconstruct the network structure even in challenging conditions of data samples available

Information Transfer in Linear Multivariate Processes Assessed through Penalized Regression Techniques: Validation and Application to Physiological Networks

The framework of information dynamics allows the dissection of the information processed in a network of multiple interacting dynamical systems into meaningful elements of computation that quantify the information generated in a target system, stored in it, transferred to it from one or more source systems, and modified in a synergistic or redundant way. The concepts of information transfer and modification have been recently formulated in the context of linear parametric modeling of vector stochastic processes, linking them to the notion of Granger causality and providing efficient tools for their computation based on the state–space (SS) representation of vector autoregressive (VAR) models. Despite their high computational reliability these tools still suffer from estimation problems which emerge, in the case of low ratio between data points available and the number of time series, when VAR identification is performed via the standard ordinary least squares (OLS). In this work we propose to replace the OLS with penalized regression performed through the Least Absolute Shrinkage and Selection Operator (LASSO), prior to computation of the measures of information transfer and information modification. First, simulating networks of several coupled Gaussian systems with complex interactions, we show that the LASSO regression allows, also in conditions of data paucity, to accurately reconstruct both the underlying network topology and the expected patterns of information transfer. Then we apply the proposed VAR-SS-LASSO approach to a challenging application context, i.e., the study of the physiological network of brain and peripheral interactions probed in humans under different conditions of rest and mental stress. Our results, which document the possibility to extract physiologically plausible patterns of interaction between the cardiovascular, respiratory and brain wave amplitudes, open the way to the use of our new analysis tools to explore the emerging field of Network Physiology in several practical applications

Connectivity Analysis in EEG Data: A Tutorial Review of the State of the Art and Emerging Trends

Understanding how different areas of the human brain communicate with each other is a crucial issue in neuroscience. The concepts of structural, functional and effective connectivity have been widely exploited to describe the human connectome, consisting of brain networks, their structural connections and functional interactions. Despite high-spatial-resolution imaging techniques such as functional magnetic resonance imaging (fMRI) being widely used to map this complex network of multiple interactions, electroencephalographic (EEG) recordings claim high temporal resolution and are thus perfectly suitable to describe either spatially distributed and temporally dynamic patterns of neural activation and connectivity. In this work, we provide a technical account and a categorization of the most-used data-driven approaches to assess brain-functional connectivity, intended as the study of the statistical dependencies between the recorded EEG signals. Different pairwise and multivariate, as well as directed and non-directed connectivity metrics are discussed with a pros-cons approach, in the time, frequency, and information-theoretic domains. The establishment of conceptual and mathematical relationships between metrics from these three frameworks, and the discussion of novel methodological approaches, will allow the reader to go deep into the problem of inferring functional connectivity in complex networks. Furthermore, emerging trends for the description of extended forms of connectivity (e.g., high-order interactions) are also discussed, along with graph-theory tools exploring the topological properties of the network of connections provided by the proposed metrics. Applications to EEG data are reviewed. In addition, the importance of source localization, and the impacts of signal acquisition and pre-processing techniques (e.g., filtering, source localization, and artifact rejection) on the connectivity estimates are recognized and discussed. By going through this review, the reader could delve deeply into the entire process of EEG pre-processing and analysis for the study of brain functional connectivity and learning, thereby exploiting novel methodologies and approaches to the problem of inferring connectivity within complex networks

Time-varying information measures: an adaptive estimation of information storage with application to brain-heart interactions

Network Physiology is a rapidly growing field of study that aims to understand how physiological systems interact to maintain health. Within the information theory framework the information storage (IS) allows to measure the regularity and predictability of a dynamic process under stationarity assumption. However, this assumption does not allow to track over time the transient pathways occurring in the dynamical activity of a physiological system. To address this limitation, we propose a time-varying approach based on the recursive least squares algorithm (RLS) for estimating IS at each time instant, in non-stationary conditions. We tested this approach in simulated time-varying dynamics and in the analysis of electroencephalographic (EEG) signals recorded from healthy volunteers and timed with the heartbeat to investigate brain-heart interactions. In simulations, we show that the proposed approach allows to track both abrupt and slow changes in the information stored in a physiological system. These changes are reflected in its evolution and variability over time. The analysis of brain-heart interactions reveals marked differences across the cardiac cycle phases of the variability of the time-varying IS. On the other hand, the average IS values exhibit a weak modulation over parieto-occiptal areas of the scalp. Our study highlights the importance of developing more advanced methods for measuring IS that account for non-stationarity in physiological systems. The proposed time-varying approach based on RLS represents a useful tool for identifying spatio-temporal dynamics within the neurocardiac system and can contribute to the understanding of brain-heart interactions

Mutual Information Rate Decomposition as a Tool to Investigate Coupled Dynamical Systems: Estimation Approaches, Simulations and Application to Physiological Signals

In this work, we present a framework for the computation of the MIR between two random processes X and Y, expressed equivalently as the sum of the individual entropy rates of X and Y minus their joint entropy rate, or as the sum of the transfer entropies from X to Y and from Y to X plus the instantaneous information shared by the processes at zero lag. After defining the theoretical formulation of the framework, different approaches for the estimation of each dynamic measure composing the MIR are provided: the linear model-based estimator relying on Gaussian data; two model-free estimators based on discretization, performed via uniform quantization through binning or rank ordering through permutations; a model-free estimator based on direct computation of the differential entropy via k-nearest neighbor searches

Multivariate correlation measures reveal structure and strength of brain–body physiological networks at rest and during mental stress

In this work, we extend to the multivariate case the classical correlation analysis used in the field of network physiology to probe dynamic interactions between organ systems in the human body. To this end, we define different correlation-based measures of the multivariate interaction (MI) within and between the brain and body subnetworks of the human physiological network, represented, respectively, by the time series of delta, theta, alpha, and beta electroencephalographic (EEG) wave amplitudes, and of heart rate, respiration amplitude, and pulse arrival time (PAT) variability (eta, rho, pi). MI is computed: (i) considering all variables in the two subnetworks to evaluate overall brain-body interactions; (ii) focusing on a single target variable and dissecting its global interaction with all other variables into contributions arising from the same subnetwork and from the other subnetwork; and (iii) considering two variables conditioned to all the others to infer the network topology. The framework is applied to the time series measured from the EEG, electrocardiographic (ECG), respiration, and blood volume pulse (BVP) signals recorded synchronously via wearable sensors in a group of healthy subjects monitored at rest and during mental arithmetic and sustained attention tasks. We find that the human physiological network is highly connected, with predominance of the links internal of each subnetwork (mainly eta-rho and delta-theta, theta-alpha, alpha-beta), but also statistically significant interactions between the two subnetworks (mainly eta-beta and eta-delta). MI values are often spatially heterogeneous across the scalp and are modulated by the physiological state, as indicated by the decrease of cardiorespiratory interactions during sustained attention and by the increase of brain-heart interactions and of brain-brain interactions at the frontal scalp regions during mental arithmetic. These findings illustrate the complex and multi-faceted structure of interactions manifested within and between different physiological systems and subsystems across different levels of mental stress

Comparison of discretization strategies for the model-free information-theoretic assessment of short-term physiological interactions

This work presents a comparison between different approaches for the model-free estimation of information-theoretic measures of the dynamic coupling between short realizations of random processes. The measures considered are the mutual information rate (MIR) between two random processes X and Y and the terms of its decomposition evidencing either the individual entropy rates of X and Y and their joint entropy rate, or the transfer entropies from X to Y and from Y to X and the instantaneous information shared by X and Y. All measures are estimated through discretization of the random variables forming the processes, performed either via uniform quantization (binning approach) or rank ordering (permutation approach). The binning and permutation approaches are compared on simulations of two coupled non-identical Hènon systems and on three datasets, including short realizations of cardiorespiratory (CR, heart period and respiration flow), cardiovascular (CV, heart period and systolic arterial pressure), and cerebrovascular (CB, mean arterial pressure and cerebral blood flow velocity) measured in different physiological conditions, i.e., spontaneous vs paced breathing or supine vs upright positions. Our results show that, with careful selection of the estimation parameters (i.e., the embedding dimension and the number of quantization levels for the binning approach), meaningful patterns of the MIR and of its components can be achieved in the analyzed systems. On physiological time series, we found that paced breathing at slow breathing rates induces less complex and more coupled CR dynamics, while postural stress leads to unbalancing of CV interactions with prevalent baroreflex coupling and to less complex pressure dynamics with preserved CB interactions. These results are better highlighted by the permutation approach, thanks to its more parsimonious representation of the discretized dynamic patterns, which allows one to explore interactions with longer memory while limiting the curse of dimensionality

Exploring the Predictability of EEG Signals Timed with the Heartbeat: A Model-Based Approach for the Temporal and Spatial Characterization of the Brain Dynamics

This study aims to provide a temporal and spatial characterization of the human brain activity related to the cardiac cycle in terms of regularity of the brain wave amplitudes measured from electroencephalographic (EEG) signals. To achieve this objective, linear autoregressive models are employed to characterize time-series of the spectral power extracted from EEG signals, timed with the heartbeat, by using a measure of predictability. The analysis is performed on four different timeseries acquired on healthy subjects in a resting state and describing the EEG spectral content over the whole frequency spectrum and within the θ, α and β bands. Our results indicate predictability values with targeted activations in the frontal and parieto-occipital brain regions, which reflect regular amplitude modulations of the brain waves at rest, and could be linked to the cortical processing of the heartbeat

Estimation of Granger causality through Artificial Neural Networks: applications to physiological systems and chaotic electronic oscillators

One of the most challenging problems in the study of complex dynamical systems is to find the statistical interdependencies among the system components. Granger causality (GC) represents one of the most employed approaches, based on modeling the system dynamics with a linear vector autoregressive (VAR) model and on evaluating the information flow between two processes in terms of prediction error variances. In its most advanced setting, GC analysis is performed through a statespace (SS) representation of the VAR model that allows to compute both conditional and unconditional forms of GC by solving only one regression problem. While this problem is typically solved through Ordinary Least Square (OLS) estimation, a viable alternative is to use Artificial Neural Networks (ANNs) implemented in a simple structure with one input and one output layer and trained in a way such that the weights matrix corresponds to the matrix of VAR parameters. In this work, we introduce an ANN combined with SS models for the computation of GC. The ANN is trained through the Stochastic Gradient Descent L1 (SGD-L1) algorithm, and a cumulative penalty inspired from penalized regression is applied to the network weights to encourage sparsity. Simulating networks of coupled Gaussian systems, we show how the combination of ANNs and SGD-L1 allows to mitigate the strong reduction in accuracy of OLS identification in settings of low ratio between number of time series points and of VAR parameters. We also report how the performances in GC estimation are influenced by the number of iterations of gradient descent and by the learning rate used for training the ANN. We recommend using some specific combinations for these parameters to optimize the performance of GC estimation. Then, the performances of ANN and OLS are compared in terms of GC magnitude and statistical significance to highlight the potential of the new approach to reconstruct causal coupling strength and network topology even in challenging conditions of data paucity. The results highlight the importance of of a proper selection of regularization parameter which determines the degree of sparsity in the estimated network. Furthermore, we apply the two approaches to real data scenarios, to study the physiological network of brain and peripheral interactions in humans under different conditions of rest and mental stress, and the effects of the newly emerged concept of remote synchronization on the information exchanged in a ring of electronic oscillators. The results highlight how ANNs provide a mesoscopic description of the information exchanged in networks of multiple interacting physiological systems, preserving the most active causal interactions between cardiovascular, respiratory and brain systems. Moreover, ANNs can reconstruct the flow of directed information in a ring of oscillators whose statistical properties can be related to those of physiological network