46,879 research outputs found
Permutation Complexity and Coupling Measures in Hidden Markov Models
In [Haruna, T. and Nakajima, K., 2011. Physica D 240, 1370-1377], the authors
introduced the duality between values (words) and orderings (permutations) as a
basis to discuss the relationship between information theoretic measures for
finite-alphabet stationary stochastic processes and their permutation
analogues. It has been used to give a simple proof of the equality between the
entropy rate and the permutation entropy rate for any finite-alphabet
stationary stochastic process and show some results on the excess entropy and
the transfer entropy for finite-alphabet stationary ergodic Markov processes.
In this paper, we extend our previous results to hidden Markov models and show
the equalities between various information theoretic complexity and coupling
measures and their permutation analogues. In particular, we show the following
two results within the realm of hidden Markov models with ergodic internal
processes: the two permutation analogues of the transfer entropy, the symbolic
transfer entropy and the transfer entropy on rank vectors, are both equivalent
to the transfer entropy if they are considered as the rates, and the directed
information theory can be captured by the permutation entropy approach.Comment: 26 page
Granger causality and transfer entropy are equivalent for Gaussian variables
Granger causality is a statistical notion of causal influence based on
prediction via vector autoregression. Developed originally in the field of
econometrics, it has since found application in a broader arena, particularly
in neuroscience. More recently transfer entropy, an information-theoretic
measure of time-directed information transfer between jointly dependent
processes, has gained traction in a similarly wide field. While it has been
recognized that the two concepts must be related, the exact relationship has
until now not been formally described. Here we show that for Gaussian
variables, Granger causality and transfer entropy are entirely equivalent, thus
bridging autoregressive and information-theoretic approaches to data-driven
causal inference.Comment: In review, Phys. Rev. Lett., Nov. 200
Informative and misinformative interactions in a school of fish
It is generally accepted that, when moving in groups, animals process
information to coordinate their motion. Recent studies have begun to apply
rigorous methods based on Information Theory to quantify such distributed
computation. Following this perspective, we use transfer entropy to quantify
dynamic information flows locally in space and time across a school of fish
during directional changes around a circular tank, i.e. U-turns. This analysis
reveals peaks in information flows during collective U-turns and identifies two
different flows: an informative flow (positive transfer entropy) based on fish
that have already turned about fish that are turning, and a misinformative flow
(negative transfer entropy) based on fish that have not turned yet about fish
that are turning. We also reveal that the information flows are related to
relative position and alignment between fish, and identify spatial patterns of
information and misinformation cascades. This study offers several
methodological contributions and we expect further application of these
methodologies to reveal intricacies of self-organisation in other animal groups
and active matter in general
- …