16,104 research outputs found
Symbolic transfer entropy rate is equal to transfer entropy rate for bivariate finite-alphabet stationary ergodic Markov processes
Transfer entropy is a measure of the magnitude and the direction of
information flow between jointly distributed stochastic processes. In recent
years, its permutation analogues are considered in the literature to estimate
the transfer entropy by counting the number of occurrences of orderings of
values, not the values themselves. It has been suggested that the method of
permutation is easy to implement, computationally low cost and robust to noise
when applying to real world time series data. In this paper, we initiate a
theoretical treatment of the corresponding rates. In particular, we consider
the transfer entropy rate and its permutation analogue, the symbolic transfer
entropy rate, and show that they are equal for any bivariate finite-alphabet
stationary ergodic Markov process. This result is an illustration of the
duality method introduced in [T. Haruna and K. Nakajima, Physica D 240, 1370
(2011)]. We also discuss the relationship among the transfer entropy rate, the
time-delayed mutual information rate and their permutation analogues.Comment: 18 page
Permutation Complexity and Coupling Measures in Hidden Markov Models
In [Haruna, T. and Nakajima, K., 2011. Physica D 240, 1370-1377], the authors
introduced the duality between values (words) and orderings (permutations) as a
basis to discuss the relationship between information theoretic measures for
finite-alphabet stationary stochastic processes and their permutation
analogues. It has been used to give a simple proof of the equality between the
entropy rate and the permutation entropy rate for any finite-alphabet
stationary stochastic process and show some results on the excess entropy and
the transfer entropy for finite-alphabet stationary ergodic Markov processes.
In this paper, we extend our previous results to hidden Markov models and show
the equalities between various information theoretic complexity and coupling
measures and their permutation analogues. In particular, we show the following
two results within the realm of hidden Markov models with ergodic internal
processes: the two permutation analogues of the transfer entropy, the symbolic
transfer entropy and the transfer entropy on rank vectors, are both equivalent
to the transfer entropy if they are considered as the rates, and the directed
information theory can be captured by the permutation entropy approach.Comment: 26 page
Symbolic local information transfer
Recently, the permutation-information theoretic approach has been used in a
broad range of research fields. In particular, in the study of highdimensional
dynamical systems, it has been shown that this approach can be effective in
characterizing global properties, including the complexity of their
spatiotemporal dynamics. Here, we show that this approach can also be applied
to reveal local spatiotemporal profiles of distributed computations existing at
each spatiotemporal point in the system. J. T. Lizier et al. have recently
introduced the concept of local information dynamics, which consists of
information storage, transfer, and modification. This concept has been
intensively studied with regard to cellular automata, and has provided
quantitative evidence of several characteristic behaviors observed in the
system. In this paper, by focusing on the local information transfer, we
demonstrate that the application of the permutation-information theoretic
approach, which introduces natural symbolization methods, makes the concept
easily extendible to systems that have continuous states. We propose measures
called symbolic local transfer entropies, and apply these measures to two test
models, the coupled map lattice (CML) system and the Bak-Sneppen model
(BS-model), to show their relevance to spatiotemporal systems that have
continuous states.Comment: 20 pages, 7 figure
Statistical Mechanics and Information-Theoretic Perspectives on Complexity in the Earth System
Peer reviewedPublisher PD
Double symbolic joint entropy in nonlinear dynamic complexity analysis
Symbolizations, the base of symbolic dynamic analysis, are classified as
global static and local dynamic approaches which are combined by joint entropy
in our works for nonlinear dynamic complexity analysis. Two global static
methods, symbolic transformations of Wessel N. symbolic entropy and base-scale
entropy, and two local ones, namely symbolizations of permutation and
differential entropy, constitute four double symbolic joint entropies that have
accurate complexity detections in chaotic models, logistic and Henon map
series. In nonlinear dynamical analysis of different kinds of heart rate
variability, heartbeats of healthy young have higher complexity than those of
the healthy elderly, and congestive heart failure (CHF) patients are lowest in
heartbeats' joint entropy values. Each individual symbolic entropy is improved
by double symbolic joint entropy among which the combination of base-scale and
differential symbolizations have best complexity analysis. Test results prove
that double symbolic joint entropy is feasible in nonlinear dynamic complexity
analysis.Comment: 7 pages, 4 figure
- …