Mixing Bandt-Pompe and Lempel-Ziv approaches: another way to analyze the complexity of continuous-states sequences

In this paper, we propose to mix the approach underlying Bandt-Pompe permutation entropy with Lempel-Ziv complexity, to design what we call Lempel-Ziv permutation complexity. The principle consists of two steps: (i) transformation of a continuous-state series that is intrinsically multivariate or arises from embedding into a sequence of permutation vectors, where the components are the positions of the components of the initial vector when re-arranged; (ii) performing the Lempel-Ziv complexity for this series of `symbols', as part of a discrete finite-size alphabet. On the one hand, the permutation entropy of Bandt-Pompe aims at the study of the entropy of such a sequence; i.e., the entropy of patterns in a sequence (e.g., local increases or decreases). On the other hand, the Lempel-Ziv complexity of a discrete-state sequence aims at the study of the temporal organization of the symbols (i.e., the rate of compressibility of the sequence). Thus, the Lempel-Ziv permutation complexity aims to take advantage of both of these methods. The potential from such a combined approach - of a permutation procedure and a complexity analysis - is evaluated through the illustration of some simulated data and some real data. In both cases, we compare the individual approaches and the combined approach.Comment: 30 pages, 4 figure

Localizing the Latent Structure Canonical Uncertainty: Entropy Profiles for Hidden Markov Models

This report addresses state inference for hidden Markov models. These models rely on unobserved states, which often have a meaningful interpretation. This makes it necessary to develop diagnostic tools for quantification of state uncertainty. The entropy of the state sequence that explains an observed sequence for a given hidden Markov chain model can be considered as the canonical measure of state sequence uncertainty. This canonical measure of state sequence uncertainty is not reflected by the classic multivariate state profiles computed by the smoothing algorithm, which summarizes the possible state sequences. Here, we introduce a new type of profiles which have the following properties: (i) these profiles of conditional entropies are a decomposition of the canonical measure of state sequence uncertainty along the sequence and makes it possible to localize this uncertainty, (ii) these profiles are univariate and thus remain easily interpretable on tree structures. We show how to extend the smoothing algorithms for hidden Markov chain and tree models to compute these entropy profiles efficiently.Comment: Submitted to Journal of Machine Learning Research; No RR-7896 (2012

Topological sequence entropy of nonautonomous dynamical systems

Let $f_{0,\infty}=\{f_n\}_{n=0}^{\infty}$ be a sequence of continuous self-maps on a compact metric space $X$. Firstly, we obtain the relations between topological sequence entropy of a nonautonomous dynamical system $(X,f_{0,\infty})$ and that of its finite-to-one extension. We then prove that the topological sequence entropy of $(X,f_{0,\infty})$ is no less than its corresponding measure sequence entropy if $X$ has finite covering dimension. Secondly, we study the supremum topological sequence entropy of $(X,f_{0,\infty})$, and confirm that it equals to that of its $n$-th compositions system if $f_{0,\infty}$ is equi-continuous; and we prove the supremum topological sequence entropy of $(X,f_{i,\infty})$ is no larger than that of $(X,f_{j,\infty})$ if $i\leq j$, and they are equal if $f_{0,\infty}$ is equi-continuous and surjective. Thirdly, we investigate the topological sequence entropy relations between $(X,f_{0,\infty})$ and $(\mathcal{M}(X),\hat{f}_{0,\infty})$ induced on the space $\mathcal{M}(X)$ of all Borel probability measures, and obtain that given any sequence, the topological sequence entropy of $(X,f_{0,\infty})$ is zero if and only if that of $(\mathcal{M}(X),\hat{f}_{0,\infty})$ is zero; the topological sequence entropy of $(X,f_{0,\infty})$ is positive if and only if that of $(\mathcal{M}(X),\hat{f}_{0,\infty})$ is infinite. By applying this result, we obtain some big differences between entropies of nonautonomous dynamical systems and that of autonomous dynamical systems. Finally, we study whether multi-sensitivity of $(X,f_{0,\infty})$ imply positive or infinite topological sequence entropy