Bits from Biology for Computational Intelligence

Computational intelligence is broadly defined as biologically-inspired computing. Usually, inspiration is drawn from neural systems. This article shows how to analyze neural systems using information theory to obtain constraints that help identify the algorithms run by such systems and the information they represent. Algorithms and representations identified information-theoretically may then guide the design of biologically inspired computing systems (BICS). The material covered includes the necessary introduction to information theory and the estimation of information theoretic quantities from neural data. We then show how to analyze the information encoded in a system about its environment, and also discuss recent methodological developments on the question of how much information each agent carries about the environment either uniquely, or redundantly or synergistically together with others. Last, we introduce the framework of local information dynamics, where information processing is decomposed into component processes of information storage, transfer, and modification -- locally in space and time. We close by discussing example applications of these measures to neural data and other complex systems

Respiratory Sinus Arrhythmia Mechanisms in Young Obese Subjects

Autonomic nervous system (ANS) activity and imbalance between its sympathetic and parasympathetic components are important factors contributing to the initiation and progression of many cardiovascular disorders related to obesity. The results on respiratory sinus arrhythmia (RSA) magnitude changes as a parasympathetic index were not straightforward in previous studies on young obese subjects. Considering the potentially unbalanced ANS regulation with impaired parasympathetic control in obese patients, the aim of this study was to compare the relative contribution of baroreflex and non-baroreflex (central) mechanisms to the origin of RSA in obese vs. control subjects. To this end, we applied a recently proposed information-theoretic methodology – partial information decomposition (PID) – to the time series of heart rate variability (HRV, computed from RR intervals in the ECG), systolic blood pressure (SBP) variability, and respiration (RESP) pattern measured in 29 obese and 29 ageand gender-matched non-obese adolescents and young adults monitored in the resting supine position and during postural and cognitive stress evoked by head-up tilt and mental arithmetic. PID was used to quantify the so-called unique information transferred from RESP to HRV and from SBP to HRV, reflecting, respectively, non-baroreflex and RESP-unrelated baroreflex HRV mechanisms, and the redundant information transferred from (RESP, SBP) to HRV, reflecting RESP-related baroreflex RSA mechanisms. Our results suggest that obesity is associated: (i) with blunted involvement of non-baroreflex RSA mechanisms, documented by the lower unique information transferred from RESP to HRV at rest; and (ii) with a reduced response to postural stress (but not to mental stress), documented by the lack of changes in the unique information transferred from RESP and SBP to HRV in obese subjects moving from supine to upright, and by a decreased redundant information transfer in obese compared to controls in the upright position. These findings were observed in the presence of an unchanged RSA magnitude measured as the high frequency (HF) power of HRV, thus suggesting that the changes in ANS imbalance related to obesity in adolescents and young adults are subtle and can be revealed by dissecting RSA mechanisms into its components during various challenges

Gradients of O-information highlight synergy and redundancy in physiological applications

The study of high order dependencies in complex systems has recently led to the introduction of statistical synergy, a novel quantity corresponding to a form of emergence in which patterns at large scales are not traceable from lower scales. As a consequence, several works in the last years dealt with the synergy and its counterpart, the redundancy. In particular, the O-information is a signed metric that measures the balance between redundant and synergistic statistical dependencies. In spite of its growing use, this metric does not provide insight about the role played by low-order scales in the formation of high order effects. To fill this gap, the framework for the computation of the O-information has been recently expanded introducing the so-called gradients of this metric, which measure the irreducible contribution of a variable (or a group of variables) to the high order informational circuits of a system. Here, we review the theory behind the O-information and its gradients and present the potential of these concepts in the field of network physiology, showing two new applications relevant to brain functional connectivity probed via functional resonance imaging and physiological interactions among the variability of heart rate, arterial pressure, respiration and cerebral blood flow

Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes

Exploiting the theory of state space models, we derive the exact expressions of the information transfer, as well as redundant and synergistic transfer, for coupled Gaussian processes observed at multiple temporal scales. All of the terms, constituting the frameworks known as interaction information decomposition and partial information decomposition, can thus be analytically obtained for different time scales from the parameters of the VAR model that fits the processes. We report the application of the proposed methodology firstly to benchmark Gaussian systems, showing that this class of systems may generate patterns of information decomposition characterized by mainly redundant or synergistic information transfer persisting across multiple time scales or even by the alternating prevalence of redundant and synergistic source interaction depending on the time scale. Then, we apply our method to an important topic in neuroscience, i.e., the detection of causal interactions in human epilepsy networks, for which we show the relevance of partial information decomposition to the detection of multiscale information transfer spreading from the seizure onset zone

Disentangling cardiovascular control mechanisms during head-down tilt via joint transfer entropy and self-entropy decompositions

A full decomposition of the predictive entropy (PE) of the spontaneous variations of the heart period (HP) given systolic arterial pressure (SAP) and respiration (R) is proposed. The PE of HP is decomposed into the joint transfer entropy (JTE) from SAP and R to HP and self-entropy (SE) of HP. The SE is the sum of three terms quantifying the synergistic/redundant contributions of HP and SAP, when taken individually and jointly, to SE and one term conditioned on HP and SAP denoted as the conditional SE (CSE) of HP given SAP and R. The JTE from SAP and R to HP is the sum of two terms attributable to SAP or R plus an extra term describing the redundant/synergistic contribution to the JTE. All quantities were computed during cardiopulmonary loading induced by -25\uc2\ub0 head-down tilt (HDT) via a multivariate linear regression approach. We found that: (i) the PE of HP decreases during HDT; (ii) the decrease of PE is attributable to a lessening of SE of HP, while the JTE from SAP and R to HP remains constant; (iii) the SE of HP is dominant over the JTE from SAP and R to HP and the CSE of HP given SAP and R is prevailing over the SE of HP due to SAP and R both in supine position and during HDT; (iv) all terms of the decompositions of JTE from SAP and R to HP and SE of HP due to SAP and R were not affected by HDT; (v) the decrease of the SE of HP during HDT was attributed to the reduction of the CSE of HP given SAP and R; (vi) redundancy of SAP and R is prevailing over synergy in the information transferred into HP both in supine position and during HDT, while in the HP information storage synergy and redundancy are more balanced. The approach suggests that the larger complexity of the cardiac control during HDT is unrelated to the baroreflex control and cardiopulmonary reflexes and may be related to central commands and/or modifications of the dynamical properties of the sinus node

A New Framework for the Time- and Frequency-Domain Assessment of High-Order Interactions in Networks of Random Processes

While the standard network description of complex systems is based on quantifying the link between pairs of system units, higher-order interactions (HOIs) involving three or more units often play a major role in governing the collective network behavior. This work introduces a new approach to quantify pairwise and HOIs for multivariate rhythmic processes interacting across multiple time scales. We define the so-called O-information rate (OIR) as a new metric to assess HOIs for multivariate time series, and present a framework to decompose the OIR into measures quantifying Granger-causal and instantaneous influences, as well as to expand all measures in the frequency domain. The framework exploits the spectral representation of vector autoregressive and state space models to assess the synergistic and redundant interaction among groups of processes, both in specific bands of interest and in the time domain after whole-band integration. Validation of the framework on simulated networks illustrates how the spectral OIR can highlight redundant and synergistic HOIs emerging at specific frequencies, which cannot be detected using time-domain measures. The applications to physiological networks described by heart period, arterial pressure and respiration variability measured in healthy subjects during a protocol of paced breathing, and to brain networks described by electrocorticographic signals acquired in an animal experiment during anesthesia, document the capability of our approach to identify informational circuits relevant to well-defined cardiovascular oscillations and brain rhythms and related to specific physiological mechanisms involving autonomic control and altered consciousness. The proposed framework allows a hierarchically-organized evaluation of timeand frequency-domain interactions in dynamic networks mapped by multivariate time series, and its high flexibility and scalability make it suitable for the investigation of networks beyond pairwise interactions in neuroscience, physiology and many other fields

Information Transfer in Linear Multivariate Processes Assessed through Penalized Regression Techniques: Validation and Application to Physiological Networks

The framework of information dynamics allows the dissection of the information processed in a network of multiple interacting dynamical systems into meaningful elements of computation that quantify the information generated in a target system, stored in it, transferred to it from one or more source systems, and modified in a synergistic or redundant way. The concepts of information transfer and modification have been recently formulated in the context of linear parametric modeling of vector stochastic processes, linking them to the notion of Granger causality and providing efficient tools for their computation based on the state–space (SS) representation of vector autoregressive (VAR) models. Despite their high computational reliability these tools still suffer from estimation problems which emerge, in the case of low ratio between data points available and the number of time series, when VAR identification is performed via the standard ordinary least squares (OLS). In this work we propose to replace the OLS with penalized regression performed through the Least Absolute Shrinkage and Selection Operator (LASSO), prior to computation of the measures of information transfer and information modification. First, simulating networks of several coupled Gaussian systems with complex interactions, we show that the LASSO regression allows, also in conditions of data paucity, to accurately reconstruct both the underlying network topology and the expected patterns of information transfer. Then we apply the proposed VAR-SS-LASSO approach to a challenging application context, i.e., the study of the physiological network of brain and peripheral interactions probed in humans under different conditions of rest and mental stress. Our results, which document the possibility to extract physiologically plausible patterns of interaction between the cardiovascular, respiratory and brain wave amplitudes, open the way to the use of our new analysis tools to explore the emerging field of Network Physiology in several practical applications

Categorizing the Role of Respiration in Cardiovascular and Cerebrovascular Variability Interactions

Objective: Respiration disturbs cardiovascular and cerebrovascular controls but its role is not fully elucidated. Methods: Respiration can be classified as a confounder if its observation reduces the strength of the causal relationship from source to target. Respiration is a suppressor if the opposite situation holds. We prove that a confounding/suppression (C/S) test can be accomplished by evaluating the sign of net redundancy/synergy balance in the predictability framework based on multivariate autoregressive modelling. In addition, we suggest that, under the hypothesis of Gaussian processes, the C/S test can be given in the transfer entropy decomposition framework as well. Experimental protocols: We applied the C/S test to variability series of respiratory movements, heart period, systolic arterial pressure, mean arterial pressure, and mean cerebral blood flow recorded in 17 pathological individuals (age: 648 yrs; 17 males) before and after induction of propofol-based general anesthesia prior to coronary artery bypass grafting, and in 13 healthy subjects (age: 278 yrs; 5 males) at rest in supine position and during head-up tilt with a table inclination of 60. Results: Respiration behaved systematically as a confounder for cardiovascular and cerebrovascular controls. In addition, its role was affected by propofol-based general anesthesia but not by a postural stimulus of limited intensity. Conclusion: The C/S test can be fruitfully exploited to categorize the role of respiration over causal variability interactions. Significance: The application of the C/S test could favor the comprehension of the role of respiration in cardiovascular and cerebrovascular regulations

Multivariate correlation measures reveal structure and strength of brain–body physiological networks at rest and during mental stress

In this work, we extend to the multivariate case the classical correlation analysis used in the field of network physiology to probe dynamic interactions between organ systems in the human body. To this end, we define different correlation-based measures of the multivariate interaction (MI) within and between the brain and body subnetworks of the human physiological network, represented, respectively, by the time series of delta, theta, alpha, and beta electroencephalographic (EEG) wave amplitudes, and of heart rate, respiration amplitude, and pulse arrival time (PAT) variability (eta, rho, pi). MI is computed: (i) considering all variables in the two subnetworks to evaluate overall brain-body interactions; (ii) focusing on a single target variable and dissecting its global interaction with all other variables into contributions arising from the same subnetwork and from the other subnetwork; and (iii) considering two variables conditioned to all the others to infer the network topology. The framework is applied to the time series measured from the EEG, electrocardiographic (ECG), respiration, and blood volume pulse (BVP) signals recorded synchronously via wearable sensors in a group of healthy subjects monitored at rest and during mental arithmetic and sustained attention tasks. We find that the human physiological network is highly connected, with predominance of the links internal of each subnetwork (mainly eta-rho and delta-theta, theta-alpha, alpha-beta), but also statistically significant interactions between the two subnetworks (mainly eta-beta and eta-delta). MI values are often spatially heterogeneous across the scalp and are modulated by the physiological state, as indicated by the decrease of cardiorespiratory interactions during sustained attention and by the increase of brain-heart interactions and of brain-brain interactions at the frontal scalp regions during mental arithmetic. These findings illustrate the complex and multi-faceted structure of interactions manifested within and between different physiological systems and subsystems across different levels of mental stress