Slope Entropy: A New Time Series Complexity Estimator Based on Both Symbolic Patterns and Amplitude Information

[EN] The development of new measures and algorithms to quantify the entropy or related concepts of a data series is a continuous effort that has brought many innovations in this regard in recent years. The ultimate goal is usually to find new methods with a higher discriminating power, more efficient, more robust to noise and artifacts, less dependent on parameters or configurations, or any other possibly desirable feature. Among all these methods, Permutation Entropy (PE) is a complexity estimator for a time series that stands out due to its many strengths, with very few weaknesses. One of these weaknesses is the PE's disregarding of time series amplitude information. Some PE algorithm modifications have been proposed in order to introduce such information into the calculations. We propose in this paper a new method, Slope Entropy (SlopEn), that also addresses this flaw but in a different way, keeping the symbolic representation of subsequences using a novel encoding method based on the slope generated by two consecutive data samples. By means of a thorough and extensive set of comparative experiments with PE and Sample Entropy (SampEn), we demonstrate that SlopEn is a very promising method with clearly a better time series classification performance than those previous methods.Cuesta Frau, D. (2019). Slope Entropy: A New Time Series Complexity Estimator Based on Both Symbolic Patterns and Amplitude Information. Entropy. 21(12):1-22. https://doi.org/10.3390/e21121167S1222112Kannathal, N., Choo, M. L., Acharya, U. R., & Sadasivan, P. K. (2005). Entropies for detection of epilepsy in EEG. Computer Methods and Programs in Biomedicine, 80(3), 187-194. doi:10.1016/j.cmpb.2005.06.012Abásolo, D., Hornero, R., Espino, P., Álvarez, D., & Poza, J. (2006). Entropy analysis of the EEG background activity in Alzheimer’s disease patients. Physiological Measurement, 27(3), 241-253. doi:10.1088/0967-3334/27/3/003Pincus, S. M., Gladstone, I. M., & Ehrenkranz, R. A. (1991). A regularity statistic for medical data analysis. Journal of Clinical Monitoring, 7(4), 335-345. doi:10.1007/bf01619355Pincus, S. M. (1991). Approximate entropy as a measure of system complexity. Proceedings of the National Academy of Sciences, 88(6), 2297-2301. doi:10.1073/pnas.88.6.2297Tsallis, C. (1988). Possible generalization of Boltzmann-Gibbs statistics. Journal of Statistical Physics, 52(1-2), 479-487. doi:10.1007/bf01016429Sinai, Y. G. (1988). About A. N. Kolmogorov’s work on the entropy of dynamical systems. Ergodic Theory and Dynamical Systems, 8(4), 501-502. doi:10.1017/s0143385700004648Richman, J. S., & Moorman, J. R. (2000). Physiological time-series analysis using approximate entropy and sample entropy. American Journal of Physiology-Heart and Circulatory Physiology, 278(6), H2039-H2049. doi:10.1152/ajpheart.2000.278.6.h2039Simons, S., Abasolo, D., & Escudero, J. (2015). Classification of Alzheimer’s disease from quadratic sample entropy of electroencephalogram. Healthcare Technology Letters, 2(3), 70-73. doi:10.1049/htl.2014.0106Simons, S., Espino, P., & Abásolo, D. (2018). Fuzzy Entropy Analysis of the Electroencephalogram in Patients with Alzheimer’s Disease: Is the Method Superior to Sample Entropy? Entropy, 20(1), 21. doi:10.3390/e20010021Cuesta-Frau, D., Novák, D., Burda, V., Molina-Picó, A., Vargas, B., Mraz, M., … Haluzik, M. (2018). Characterization of Artifact Influence on the Classification of Glucose Time Series Using Sample Entropy Statistics. Entropy, 20(11), 871. doi:10.3390/e20110871Sheng Lu, Xinnian Chen, Kanters, J. K., Solomon, I. C., & Chon, K. H. (2008). Automatic Selection of the Threshold Value $r$ for Approximate Entropy. IEEE Transactions on Biomedical Engineering, 55(8), 1966-1972. doi:10.1109/tbme.2008.919870Costa, M., Goldberger, A. L., & Peng, C.-K. (2005). Multiscale entropy analysis of biological signals. Physical Review E, 71(2). doi:10.1103/physreve.71.021906Wu, S.-D., Wu, C.-W., Lin, S.-G., Lee, K.-Y., & Peng, C.-K. (2014). Analysis of complex time series using refined composite multiscale entropy. Physics Letters A, 378(20), 1369-1374. doi:10.1016/j.physleta.2014.03.034Bandt, C., & Pompe, B. (2002). Permutation Entropy: A Natural Complexity Measure for Time Series. Physical Review Letters, 88(17). doi:10.1103/physrevlett.88.174102Aboy, M., Hornero, R., Abasolo, D., & Alvarez, D. (2006). Interpretation of the Lempel-Ziv Complexity Measure in the Context of Biomedical Signal Analysis. IEEE Transactions on Biomedical Engineering, 53(11), 2282-2288. doi:10.1109/tbme.2006.883696Cuesta-Frau, D., Murillo-Escobar, J. P., Orrego, D. A., & Delgado-Trejos, E. (2019). Embedded Dimension and Time Series Length. Practical Influence on Permutation Entropy and Its Applications. Entropy, 21(4), 385. doi:10.3390/e21040385Li, D., Liang, Z., Wang, Y., Hagihira, S., Sleigh, J. W., & Li, X. (2012). Parameter selection in permutation entropy for an electroencephalographic measure of isoflurane anesthetic drug effect. Journal of Clinical Monitoring and Computing, 27(2), 113-123. doi:10.1007/s10877-012-9419-0Cuesta–Frau, D., Varela–Entrecanales, M., Molina–Picó, A., & Vargas, B. (2018). Patterns with Equal Values in Permutation Entropy: Do They Really Matter for Biosignal Classification? Complexity, 2018, 1-15. doi:10.1155/2018/1324696Riedl, M., Müller, A., & Wessel, N. (2013). Practical considerations of permutation entropy. The European Physical Journal Special Topics, 222(2), 249-262. doi:10.1140/epjst/e2013-01862-7Zunino, L., Olivares, F., Scholkmann, F., & Rosso, O. A. (2017). Permutation entropy based time series analysis: Equalities in the input signal can lead to false conclusions. Physics Letters A, 381(22), 1883-1892. doi:10.1016/j.physleta.2017.03.052Liu, T., Yao, W., Wu, M., Shi, Z., Wang, J., & Ning, X. (2017). Multiscale permutation entropy analysis of electrocardiogram. Physica A: Statistical Mechanics and its Applications, 471, 492-498. doi:10.1016/j.physa.2016.11.102Gao, Y., Villecco, F., Li, M., & Song, W. (2017). Multi-Scale Permutation Entropy Based on Improved LMD and HMM for Rolling Bearing Diagnosis. Entropy, 19(4), 176. doi:10.3390/e19040176Azami, H., & Escudero, J. (2016). Amplitude-aware permutation entropy: Illustration in spike detection and signal segmentation. Computer Methods and Programs in Biomedicine, 128, 40-51. doi:10.1016/j.cmpb.2016.02.008Fadlallah, B., Chen, B., Keil, A., & Príncipe, J. (2013). Weighted-permutation entropy: A complexity measure for time series incorporating amplitude information. Physical Review E, 87(2). doi:10.1103/physreve.87.022911Xiao-Feng, L., & Yue, W. (2009). Fine-grained permutation entropy as a measure of natural complexity for time series. Chinese Physics B, 18(7), 2690-2695. doi:10.1088/1674-1056/18/7/011Cuesta–Frau, D. (2019). Permutation entropy: Influence of amplitude information on time series classification performance. Mathematical Biosciences and Engineering, 16(6), 6842-6857. doi:10.3934/mbe.2019342Koski, A., Juhola, M., & Meriste, M. (1995). Syntactic recognition of ECG signals by attributed finite automata. Pattern Recognition, 28(12), 1927-1940. doi:10.1016/0031-3203(95)00052-6Koski, A. (1996). Primitive coding of structural ECG features. Pattern Recognition Letters, 17(11), 1215-1222. doi:10.1016/0167-8655(96)00079-7Lake, D. E., Richman, J. S., Griffin, M. P., & Moorman, J. R. (2002). Sample entropy analysis of neonatal heart rate variability. American Journal of Physiology-Regulatory, Integrative and Comparative Physiology, 283(3), R789-R797. doi:10.1152/ajpregu.00069.2002Wessel, N., Ziehmann, C., Kurths, J., Meyerfeldt, U., Schirdewan, A., & Voss, A. (2000). Short-term forecasting of life-threatening cardiac arrhythmias based on symbolic dynamics and finite-time growth rates. Physical Review E, 61(1), 733-739. doi:10.1103/physreve.61.733Cysarz, D., Lange, S., Matthiessen, P. F., & Leeuwen, P. van. (2007). Regular heartbeat dynamics are associated with cardiac health. American Journal of Physiology-Regulatory, Integrative and Comparative Physiology, 292(1), R368-R372. doi:10.1152/ajpregu.00161.2006Cuesta-Frau, D., Molina-Picó, A., Vargas, B., & González, P. (2019). Permutation Entropy: Enhancing Discriminating Power by Using Relative Frequencies Vector of Ordinal Patterns Instead of Their Shannon Entropy. Entropy, 21(10), 1013. doi:10.3390/e21101013Mayer, C. C., Bachler, M., Hörtenhuber, M., Stocker, C., Holzinger, A., & Wassertheurer, S. (2014). Selection of entropy-measure parameters for knowledge discovery in heart rate variability data. BMC Bioinformatics, 15(S6). doi:10.1186/1471-2105-15-s6-s2Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., … Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet. Circulation, 101(23). doi:10.1161/01.cir.101.23.e215Iyengar, N., Peng, C. K., Morin, R., Goldberger, A. L., & Lipsitz, L. A. (1996). Age-related alterations in the fractal scaling of cardiac interbeat interval dynamics. American Journal of Physiology-Regulatory, Integrative and Comparative Physiology, 271(4), R1078-R1084. doi:10.1152/ajpregu.1996.271.4.r1078Bagnall, A., Lines, J., Bostrom, A., Large, J., & Keogh, E. (2016). The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Mining and Knowledge Discovery, 31(3), 606-660. doi:10.1007/s10618-016-0483-9Flood, M. W., Jensen, B. R., Malling, A.-S., & Lowery, M. M. (2019). Increased EMG intermuscular coherence and reduced signal complexity in Parkinson’s disease. Clinical Neurophysiology, 130(2), 259-269. doi:10.1016/j.clinph.2018.10.023Tang, X., Zhang, X., Gao, X., Chen, X., & Zhou, P. (2018). A Novel Interpretation of Sample Entropy in Surface Electromyographic Examination of Complex Neuromuscular Alternations in Subacute and Chronic Stroke. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 26(9), 1878-1888. doi:10.1109/tnsre.2018.2864317Zhu, X., Zhang, X., Tang, X., Gao, X., & Chen, X. (2017). Re-Evaluating Electromyogram–Force Relation in Healthy Biceps Brachii Muscles Using Complexity Measures. Entropy, 19(11), 624. doi:10.3390/e19110624Bingham, A., Arjunan, S., Jelfs, B., & Kumar, D. (2017). Normalised Mutual Information of High-Density Surface Electromyography during Muscle Fatigue. Entropy, 19(12), 697. doi:10.3390/e19120697Montesinos, L., Castaldo, R., & Pecchia, L. (2018). On the use of approximate entropy and sample entropy with centre of pressure time-series. Journal of NeuroEngineering and Rehabilitation, 15(1). doi:10.1186/s12984-018-0465-9Bubble Entropy: An Entropy Almost Free of Parameters. (2017). IEEE Transactions on Biomedical Engineering, 64(11), 2711-2718. doi:10.1109/tbme.2017.2664105Li, D., Li, X., Liang, Z., Voss, L. J., & Sleigh, J. W. (2010). Multiscale permutation entropy analysis of EEG recordings during sevoflurane anesthesia. Journal of Neural Engineering, 7(4), 046010. doi:10.1088/1741-2560/7/4/046010Costa, M., Goldberger, A. L., & Peng, C.-K. (2002). Multiscale Entropy Analysis of Complex Physiologic Time Series. Physical Review Letters, 89(6). doi:10.1103/physrevlett.89.068102Hari, V. N., Anand, G. V., Premkumar, A. B., & Madhukumar, A. S. (2012). Design and performance analysis of a signal detector based on suprathreshold stochastic resonance. Signal Processing, 92(7), 1745-1757. doi:10.1016/j.sigpro.2012.01.01

Safety Evaluation of Critical Applications Distributed on TDMA-Based Networks

Critical embedded systems have to provide a high level of dependability. In automotive domain, for example, TDMA protocols are largely recommended because of their deterministic behavior. Nevertheless, under the transient environmental perturbations, the loss of communication cycles may occur with a certain probability and, consequently, the system may fail. This paper analyzes the impact of the transient perturbations (especially due to Electromagnetic Interferences) on the dependability of systems distributed on TDMA-based networks. The dependability of such system is modeled as that of "consecutive-k-out-of-n:F" systems and we provide a efficient way for its evaluation

Exact Failure Frequency Calculations for Extended Systems

This paper shows how the steady-state availability and failure frequency can be calculated in a single pass for very large systems, when the availability is expressed as a product of matrices. We apply the general procedure to $k$-out-of-$n$:G and linear consecutive $k$-out-of-$n$:F systems, and to a simple ladder network in which each edge and node may fail. We also give the associated generating functions when the components have identical availabilities and failure rates. For large systems, the failure rate of the whole system is asymptotically proportional to its size. This paves the way to ready-to-use formulae for various architectures, as well as proof that the differential operator approach to failure frequency calculations is very useful and straightforward

Forbidden patterns and shift systems

The scope of this paper is two-fold. First, to present to the researchers in combinatorics an interesting implementation of permutations avoiding generalized patterns in the framework of discrete-time dynamical systems. Indeed, the orbits generated by piecewise monotone maps on one-dimensional intervals have forbidden order patterns, i.e., order patterns that do not occur in any orbit. The allowed patterns are then those patterns avoiding the so-called forbidden root patterns and their shifted patterns. The second scope is to study forbidden patterns in shift systems, which are universal models in information theory, dynamical systems and stochastic processes. Due to its simple structure, shift systems are accessible to a more detailed analysis and, at the same time, exhibit all important properties of low-dimensional chaotic dynamical systems (e.g., sensitivity to initial conditions, strong mixing and a dense set of periodic points), allowing to export the results to other dynamical systems via order-isomorphisms.Comment: 21 pages, expanded Section 5 and corrected Propositions 3 and

Identification of criticality in neuronal avalanches: II. A theoretical and empirical investigation of the Driven case

The observation of apparent power laws in neuronal systems has led to the suggestion that the brain is at, or close to, a critical state and may be a self-organised critical system. Within the framework of self-organised criticality a separation of timescales is thought to be crucial for the observation of power-law dynamics and computational models are often constructed with this property. However, this is not necessarily a characteristic of physiological neural networks—external input does not only occur when the network is at rest/a steady state. In this paper we study a simple neuronal network model driven by a continuous external input (i.e. the model does not have an explicit separation of timescales from seeding the system only when in the quiescent state) and analytically tuned to operate in the region of a critical state (it reaches the critical regime exactly in the absence of input—the case studied in the companion paper to this article). The system displays avalanche dynamics in the form of cascades of neuronal firing separated by periods of silence. We observe partial scale-free behaviour in the distribution of avalanche size for low levels of external input. We analytically derive the distributions of waiting times and investigate their temporal behaviour in relation to different levels of external input, showing that the system’s dynamics can exhibit partial long-range temporal correlations. We further show that as the system approaches the critical state by two alternative ‘routes’, different markers of criticality (partial scale-free behaviour and long-range temporal correlations) are displayed. This suggests that signatures of criticality exhibited by a particular system in close proximity to a critical state are dependent on the region in parameter space at which the system (currently) resides

Semantic Stability in Social Tagging Streams

One potential disadvantage of social tagging systems is that due to the lack of a centralized vocabulary, a crowd of users may never manage to reach a consensus on the description of resources (e.g., books, users or songs) on the Web. Yet, previous research has provided interesting evidence that the tag distributions of resources may become semantically stable over time as more and more users tag them. At the same time, previous work has raised an array of new questions such as: (i) How can we assess the semantic stability of social tagging systems in a robust and methodical way? (ii) Does semantic stabilization of tags vary across different social tagging systems and ultimately, (iii) what are the factors that can explain semantic stabilization in such systems? In this work we tackle these questions by (i) presenting a novel and robust method which overcomes a number of limitations in existing methods, (ii) empirically investigating semantic stabilization processes in a wide range of social tagging systems with distinct domains and properties and (iii) detecting potential causes for semantic stabilization, specifically imitation behavior, shared background knowledge and intrinsic properties of natural language. Our results show that tagging streams which are generated by a combination of imitation dynamics and shared background knowledge exhibit faster and higher semantic stability than tagging streams which are generated via imitation dynamics or natural language streams alone

Planets Transiting Non-Eclipsing Binaries

The majority of binary stars do not eclipse. Current searches for transiting circumbinary planets concentrate on eclipsing binaries, and are therefore restricted to a small fraction of potential hosts. We investigate the concept of finding planets transiting non-eclipsing binaries, whose geometry would require mutually inclined planes. Using an N-body code we explore how the number and sequence of transits vary as functions of observing time and orbital parameters. The concept is then generalised thanks to a suite of simulated circumbinary systems. Binaries are constructed from RV surveys of the solar neighbourhood. They are then populated with orbiting gas giants, drawn from a range of distributions. The binary population is shown to be compatible with the Kepler eclipsing binary catalogue, indicating that the properties of binaries may be as universal as the initial mass function. These synthetic systems produce transiting circumbinary planets occurring on both eclipsing and non-eclipsing binaries. Simulated planets transiting eclipsing binaries are compared with published Kepler detections. We obtain 1) that planets transiting non-eclipsing binaries probably exist in the Kepler data, 2) that observational biases alone cannot account for the observed over-density of circumbinary planets near the stability limit, implying a physical pile-up, and 3) that the distributions of gas giants orbiting single and binary stars are likely different. Estimating the frequency of circumbinary planets is degenerate with the spread in mutual inclination. Only a minimum occurrence rate can be produced, which we find to be compatible with 9%. Searching for inclined circumbinary planets may significantly increase the population of known objects and will test our conclusions. Their existence, or absence, will reveal the true occurrence rate and help develop circumbinary planet formation theories.Comment: 19 pages, 14 figures, accepted August 2014 to A&A, minor changes to previous arXiv versio