43,396 research outputs found
Permutation Complexity and Coupling Measures in Hidden Markov Models
In [Haruna, T. and Nakajima, K., 2011. Physica D 240, 1370-1377], the authors
introduced the duality between values (words) and orderings (permutations) as a
basis to discuss the relationship between information theoretic measures for
finite-alphabet stationary stochastic processes and their permutation
analogues. It has been used to give a simple proof of the equality between the
entropy rate and the permutation entropy rate for any finite-alphabet
stationary stochastic process and show some results on the excess entropy and
the transfer entropy for finite-alphabet stationary ergodic Markov processes.
In this paper, we extend our previous results to hidden Markov models and show
the equalities between various information theoretic complexity and coupling
measures and their permutation analogues. In particular, we show the following
two results within the realm of hidden Markov models with ergodic internal
processes: the two permutation analogues of the transfer entropy, the symbolic
transfer entropy and the transfer entropy on rank vectors, are both equivalent
to the transfer entropy if they are considered as the rates, and the directed
information theory can be captured by the permutation entropy approach.Comment: 26 page
Symbolic transfer entropy rate is equal to transfer entropy rate for bivariate finite-alphabet stationary ergodic Markov processes
Transfer entropy is a measure of the magnitude and the direction of
information flow between jointly distributed stochastic processes. In recent
years, its permutation analogues are considered in the literature to estimate
the transfer entropy by counting the number of occurrences of orderings of
values, not the values themselves. It has been suggested that the method of
permutation is easy to implement, computationally low cost and robust to noise
when applying to real world time series data. In this paper, we initiate a
theoretical treatment of the corresponding rates. In particular, we consider
the transfer entropy rate and its permutation analogue, the symbolic transfer
entropy rate, and show that they are equal for any bivariate finite-alphabet
stationary ergodic Markov process. This result is an illustration of the
duality method introduced in [T. Haruna and K. Nakajima, Physica D 240, 1370
(2011)]. We also discuss the relationship among the transfer entropy rate, the
time-delayed mutual information rate and their permutation analogues.Comment: 18 page
Mixing Bandt-Pompe and Lempel-Ziv approaches: another way to analyze the complexity of continuous-states sequences
In this paper, we propose to mix the approach underlying Bandt-Pompe
permutation entropy with Lempel-Ziv complexity, to design what we call
Lempel-Ziv permutation complexity. The principle consists of two steps: (i)
transformation of a continuous-state series that is intrinsically multivariate
or arises from embedding into a sequence of permutation vectors, where the
components are the positions of the components of the initial vector when
re-arranged; (ii) performing the Lempel-Ziv complexity for this series of
`symbols', as part of a discrete finite-size alphabet. On the one hand, the
permutation entropy of Bandt-Pompe aims at the study of the entropy of such a
sequence; i.e., the entropy of patterns in a sequence (e.g., local increases or
decreases). On the other hand, the Lempel-Ziv complexity of a discrete-state
sequence aims at the study of the temporal organization of the symbols (i.e.,
the rate of compressibility of the sequence). Thus, the Lempel-Ziv permutation
complexity aims to take advantage of both of these methods. The potential from
such a combined approach - of a permutation procedure and a complexity analysis
- is evaluated through the illustration of some simulated data and some real
data. In both cases, we compare the individual approaches and the combined
approach.Comment: 30 pages, 4 figure
Anomaly Detection in Paleoclimate Records using Permutation Entropy
Permutation entropy techniques can be useful in identifying anomalies in
paleoclimate data records, including noise, outliers, and post-processing
issues. We demonstrate this using weighted and unweighted permutation entropy
of water-isotope records in a deep polar ice core. In one region of these
isotope records, our previous calculations revealed an abrupt change in the
complexity of the traces: specifically, in the amount of new information that
appeared at every time step. We conjectured that this effect was due to noise
introduced by an older laboratory instrument. In this paper, we validate that
conjecture by re-analyzing a section of the ice core using a more-advanced
version of the laboratory instrument. The anomalous noise levels are absent
from the permutation entropy traces of the new data. In other sections of the
core, we show that permutation entropy techniques can be used to identify
anomalies in the raw data that are not associated with climatic or
glaciological processes, but rather effects occurring during field work,
laboratory analysis, or data post-processing. These examples make it clear that
permutation entropy is a useful forensic tool for identifying sections of data
that require targeted re-analysis---and can even be useful in guiding that
analysis.Comment: 15 pages, 7 figure
Permutation Complexity via Duality between Values and Orderings
We study the permutation complexity of finite-state stationary stochastic
processes based on a duality between values and orderings between values.
First, we establish a duality between the set of all words of a fixed length
and the set of all permutations of the same length. Second, on this basis, we
give an elementary alternative proof of the equality between the permutation
entropy rate and the entropy rate for a finite-state stationary stochastic
processes first proved in [Amigo, J.M., Kennel, M. B., Kocarev, L., 2005.
Physica D 210, 77-95]. Third, we show that further information on the
relationship between the structure of values and the structure of orderings for
finite-state stationary stochastic processes beyond the entropy rate can be
obtained from the established duality. In particular, we prove that the
permutation excess entropy is equal to the excess entropy, which is a measure
of global correlation present in a stationary stochastic process, for
finite-state stationary ergodic Markov processes.Comment: 26 page
On wind Turbine failure detection from measurements of phase currents: a permutation entropy approach
This article presents the applicability of Permutation Entropy based
complexity measure of a time series for detection of fault in wind turbines. A
set of electrical data from one faulty and one healthy wind turbine were
analysed using traditional FastFourier analysis in addition to Permutation
Entropy analysis to compare the complexity index of phase currents of the two
turbines over time. The 4 seconds length data set did not reveal any low
frequency in the spectra of currents, neither did they show any meaningful
differences of spectrum between the two turbine currents. Permutation Entropy
analysis of the current waveforms of same phases for the two turbines are found
to have different complexity values over time, one of them being clearly higher
than the other. The work of Yan et. al. in has found that higher entropy values
related to thepresence of failure in rotary machines in his study. Following
this track, further efforts will be put into relating the entropy difference
found in our study to possible presence of failure in one of the wind energy
conversion systems
Numerical and experimental study of the effects of noise on the permutation entropy
We analyze the effects of noise on the permutation entropy of dynamical
systems. We take as numerical examples the logistic map and the R\"ossler
system. Upon varying the noise strengthfaster, we find a transition from an
almost-deterministic regime, where the permutation entropy grows slower than
linearly with the pattern dimension, to a noise-dominated regime, where the
permutation entropy grows faster than linearly with the pattern dimension. We
perform the same analysis on experimental time-series by considering the
stochastic spiking output of a semiconductor laser with optical feedback.
Because of the experimental conditions, the dynamics is found to be always in
the noise-dominated regime. Nevertheless, the analysis allows to detect
regularities of the underlying dynamics. By comparing the results of these
three different examples, we discuss the possibility of determining from a time
series whether the underlying dynamics is dominated by noise or not
- …