Using the Information Provided by Forbidden Ordinal Patterns in Permutation Entropy to Reinforce Time Series Discrimination Capabilities

[EN] Despite its widely tested and proven usefulness, there is still room for improvement in the basic permutation entropy (PE) algorithm, as several subsequent studies have demonstrated in recent years. Some of these new methods try to address the well-known PE weaknesses, such as its focus only on ordinal and not on amplitude information, and the possible detrimental impact of equal values found in subsequences. Other new methods address less specific weaknesses, such as the PE results¿ dependence on input parameter values, a common problem found in many entropy calculation methods. The lack of discriminating power among classes in some cases is also a generic problem when entropy measures are used for data series classification. This last problem is the one specifically addressed in the present study. Toward that purpose, the classification performance of the standard PE method was first assessed by conducting several time series classification tests over a varied and diverse set of data. Then, this performance was reassessed using a new Shannon Entropy normalisation scheme proposed in this paper: divide the relative frequencies in PE by the number of different ordinal patterns actually found in the time series, instead of by the theoretically expected number. According to the classification accuracy obtained, this last approach exhibited a higher class discriminating power. It was capable of finding significant differences in six out of seven experimental datasets¿whereas the standard PE method only did in four¿and it also had better classification accuracy. It can be concluded that using the additional information provided by the number of forbidden/found patterns, it is possible to achieve a higher discriminating power than using the classical PE normalisation method. The resulting algorithm is also very similar to that of PE and very easy to implement.Cuesta Frau, D. (2020). Using the Information Provided by Forbidden Ordinal Patterns in Permutation Entropy to Reinforce Time Series Discrimination Capabilities. Entropy. 22(5):1-17. https://doi.org/10.3390/e22050494S117225Bandt, C., & Pompe, B. (2002). Permutation Entropy: A Natural Complexity Measure for Time Series. Physical Review Letters, 88(17). doi:10.1103/physrevlett.88.174102Zanin, M., Zunino, L., Rosso, O. A., & Papo, D. (2012). Permutation Entropy and Its Main Biomedical and Econophysics Applications: A Review. Entropy, 14(8), 1553-1577. doi:10.3390/e14081553Li, J., Yan, J., Liu, X., & Ouyang, G. (2014). Using Permutation Entropy to Measure the Changes in EEG Signals During Absence Seizures. Entropy, 16(6), 3049-3061. doi:10.3390/e16063049Ravelo-García, A., Navarro-Mesa, J., Casanova-Blancas, U., Martin-Gonzalez, S., Quintana-Morales, P., Guerra-Moreno, I., … Hernández-Pérez, E. (2015). Application of the Permutation Entropy over the Heart Rate Variability for the Improvement of Electrocardiogram-based Sleep Breathing Pause Detection. Entropy, 17(3), 914-927. doi:10.3390/e17030914Cuesta-Frau, D., Miró-Martínez, P., Oltra-Crespo, S., Jordán-Núñez, J., Vargas, B., González, P., & Varela-Entrecanales, M. (2018). Model Selection for Body Temperature Signal Classification Using Both Amplitude and Ordinality-Based Entropy Measures. Entropy, 20(11), 853. doi:10.3390/e20110853Cuesta–Frau, D., Miró–Martínez, P., Oltra–Crespo, S., Jordán–Núñez, J., Vargas, B., & Vigil, L. (2018). Classification of glucose records from patients at diabetes risk using a combined permutation entropy algorithm. Computer Methods and Programs in Biomedicine, 165, 197-204. doi:10.1016/j.cmpb.2018.08.018Gao, Y., Villecco, F., Li, M., & Song, W. (2017). Multi-Scale Permutation Entropy Based on Improved LMD and HMM for Rolling Bearing Diagnosis. Entropy, 19(4), 176. doi:10.3390/e19040176Wang, X., Si, S., Wei, Y., & Li, Y. (2019). The Optimized Multi-Scale Permutation Entropy and Its Application in Compound Fault Diagnosis of Rotating Machinery. Entropy, 21(2), 170. doi:10.3390/e21020170Wang, Q. C., Song, W. Q., & Liang, J. K. (2014). Road Flatness Detection Using Permutation Entropy (PE). Applied Mechanics and Materials, 721, 420-423. doi:10.4028/www.scientific.net/amm.721.420Glynn, C. C., & Konstantinou, K. I. (2016). Reduction of randomness in seismic noise as a short-term precursor to a volcanic eruption. Scientific Reports, 6(1). doi:10.1038/srep37733Zhang, Y., & Shang, P. (2017). Permutation entropy analysis of financial time series based on Hill’s diversity number. Communications in Nonlinear Science and Numerical Simulation, 53, 288-298. doi:10.1016/j.cnsns.2017.05.003Fadlallah, B., Chen, B., Keil, A., & Príncipe, J. (2013). Weighted-permutation entropy: A complexity measure for time series incorporating amplitude information. Physical Review E, 87(2). doi:10.1103/physreve.87.022911Xiao-Feng, L., & Yue, W. (2009). Fine-grained permutation entropy as a measure of natural complexity for time series. Chinese Physics B, 18(7), 2690-2695. doi:10.1088/1674-1056/18/7/011Azami, H., & Escudero, J. (2016). Amplitude-aware permutation entropy: Illustration in spike detection and signal segmentation. Computer Methods and Programs in Biomedicine, 128, 40-51. doi:10.1016/j.cmpb.2016.02.008Cuesta–Frau, D. (2019). Permutation entropy: Influence of amplitude information on time series classification performance. Mathematical Biosciences and Engineering, 16(6), 6842-6857. doi:10.3934/mbe.2019342Bian, C., Qin, C., Ma, Q. D. Y., & Shen, Q. (2012). Modified permutation-entropy analysis of heartbeat dynamics. Physical Review E, 85(2). doi:10.1103/physreve.85.021906Cuesta–Frau, D., Varela–Entrecanales, M., Molina–Picó, A., & Vargas, B. (2018). Patterns with Equal Values in Permutation Entropy: Do They Really Matter for Biosignal Classification? Complexity, 2018, 1-15. doi:10.1155/2018/1324696Zunino, L., Olivares, F., Scholkmann, F., & Rosso, O. A. (2017). Permutation entropy based time series analysis: Equalities in the input signal can lead to false conclusions. Physics Letters A, 381(22), 1883-1892. doi:10.1016/j.physleta.2017.03.052Bubble Entropy: An Entropy Almost Free of Parameters. (2017). IEEE Transactions on Biomedical Engineering, 64(11), 2711-2718. doi:10.1109/tbme.2017.2664105Amigó, J. (2010). Permutation Complexity in Dynamical Systems. Springer Series in Synergetics. doi:10.1007/978-3-642-04084-9Amigó, J. M., Kocarev, L., & Szczepanski, J. (2006). Order patterns and chaos. Physics Letters A, 355(1), 27-31. doi:10.1016/j.physleta.2006.01.093Zunino, L., Zanin, M., Tabak, B. M., Pérez, D. G., & Rosso, O. A. (2009). Forbidden patterns, permutation entropy and stock market inefficiency. Physica A: Statistical Mechanics and its Applications, 388(14), 2854-2864. doi:10.1016/j.physa.2009.03.042Little, D. J., & Kane, D. M. (2017). Permutation entropy with vector embedding delays. Physical Review E, 96(6). doi:10.1103/physreve.96.062205Riedl, M., Müller, A., & Wessel, N. (2013). Practical considerations of permutation entropy. The European Physical Journal Special Topics, 222(2), 249-262. doi:10.1140/epjst/e2013-01862-7Rosso, O. A., Carpi, L. C., Saco, P. M., Gómez Ravetti, M., Plastino, A., & Larrondo, H. A. (2012). Causality and the entropy–complexity plane: Robustness and missing ordinal patterns. Physica A: Statistical Mechanics and its Applications, 391(1-2), 42-55. doi:10.1016/j.physa.2011.07.030Cuesta-Frau, D., Murillo-Escobar, J. P., Orrego, D. A., & Delgado-Trejos, E. (2019). Embedded Dimension and Time Series Length. Practical Influence on Permutation Entropy and Its Applications. Entropy, 21(4), 385. doi:10.3390/e21040385KumarSingh, B., Verma, K., & S. Thoke, A. (2015). Investigations on Impact of Feature Normalization Techniques on Classifier's Performance in Breast Tumor Classification. International Journal of Computer Applications, 116(19), 11-15. doi:10.5120/20443-2793Talukder, B., W. Hipel, K., & W. vanLoon, G. (2017). Developing Composite Indicators for Agricultural Sustainability Assessment: Effect of Normalization and Aggregation Techniques. Resources, 6(4), 66. doi:10.3390/resources6040066Tofallis, C. (2014). Add or Multiply? A Tutorial on Ranking and Choosing with Multiple Criteria. INFORMS Transactions on Education, 14(3), 109-119. doi:10.1287/ited.2013.0124Henry, M., & Judge, G. (2019). Permutation Entropy and Information Recovery in Nonlinear Dynamic Economic Time Series. Econometrics, 7(1), 10. doi:10.3390/econometrics7010010Zanin, M. (2008). Forbidden patterns in financial time series. Chaos: An Interdisciplinary Journal of Nonlinear Science, 18(1), 013119. doi:10.1063/1.2841197Cuesta-Frau, D., Molina-Picó, A., Vargas, B., & González, P. (2019). Permutation Entropy: Enhancing Discriminating Power by Using Relative Frequencies Vector of Ordinal Patterns Instead of Their Shannon Entropy. Entropy, 21(10), 1013. doi:10.3390/e21101013Andrzejak, R. G., Lehnertz, K., Mormann, F., Rieke, C., David, P., & Elger, C. E. (2001). Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state. Physical Review E, 64(6). doi:10.1103/physreve.64.061907Bellegdi, S. A., & Arafat, S. M. A. (2017). Automatic Detection of Epilepsy Using EEG Energy and Frequency Bands. International Journal of Applied Mathematics, Electronics and Computers, 1(SpecialIssue), 36-41. doi:10.18100/ijamec.2017specialissue30468Hussain, L., Aziz, W., Alowibdi, J. S., Habib, N., Rafique, M., Saeed, S., & Kazmi, S. Z. H. (2017). Symbolic time series analysis of electroencephalographic (EEG) epileptic seizure and brain dynamics with eye-open and eye-closed subjects during resting states. Journal of Physiological Anthropology, 36(1). doi:10.1186/s40101-017-0136-8Cuesta–Frau, D., Miró–Martínez, P., Jordán Núñez, J., Oltra–Crespo, S., & Molina Picó, A. (2017). Noisy EEG signals classification based on entropy metrics. Performance assessment using first and second generation statistics. Computers in Biology and Medicine, 87, 141-151. doi:10.1016/j.compbiomed.2017.05.028Sharmila, A. (2018). Epilepsy detection from EEG signals: a review. Journal of Medical Engineering & Technology, 42(5), 368-380. doi:10.1080/03091902.2018.1513576Molina-Picó, A., Cuesta-Frau, D., Aboy, M., Crespo, C., Miró-Martínez, P., & Oltra-Crespo, S. (2011). Comparative study of approximate entropy and sample entropy robustness to spikes. Artificial Intelligence in Medicine, 53(2), 97-106. doi:10.1016/j.artmed.2011.06.007Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., … Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet. Circulation, 101(23). doi:10.1161/01.cir.101.23.e215Hausdorff, J. M., Purdon, P. L., Peng, C. K., Ladin, Z., Wei, J. Y., & Goldberger, A. L. (1996). Fractal dynamics of human gait: stability of long-range correlations in stride interval fluctuations. Journal of Applied Physiology, 80(5), 1448-1457. doi:10.1152/jappl.1996.80.5.1448Baumert, M., Czippelova, B., Ganesan, A., Schmidt, M., Zaunseder, S., & Javorka, M. (2014). Entropy Analysis of RR and QT Interval Variability during Orthostatic and Mental Stress in Healthy Subjects. Entropy, 16(12), 6384-6393. doi:10.3390/e16126384Xia, Y., Yang, L., Zunino, L., Shi, H., Zhuang, Y., & Liu, C. (2018). Application of Permutation Entropy and Permutation Min-Entropy in Multiple Emotional States Analysis of RRI Time Series. Entropy, 20(3), 148. doi:10.3390/e20030148Iyengar, N., Peng, C. K., Morin, R., Goldberger, A. L., & Lipsitz, L. A. (1996). Age-related alterations in the fractal scaling of cardiac interbeat interval dynamics. American Journal of Physiology-Regulatory, Integrative and Comparative Physiology, 271(4), R1078-R1084. doi:10.1152/ajpregu.1996.271.4.r1078Liu, C., Li, K., Zhao, L., Liu, F., Zheng, D., Liu, C., & Liu, S. (2013). Analysis of heart rate variability using fuzzy measure entropy. Computers in Biology and Medicine, 43(2), 100-108. doi:10.1016/j.compbiomed.2012.11.005Bugenhagen, S. M., Cowley, A. W., & Beard, D. A. (2010). Identifying physiological origins of baroreflex dysfunction in salt-sensitive hypertension in the Dahl SS rat. Physiological Genomics, 42(1), 23-41. doi:10.1152/physiolgenomics.00027.2010Bagnall, A., Lines, J., Bostrom, A., Large, J., & Keogh, E. (2016). The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Mining and Knowledge Discovery, 31(3), 606-660. doi:10.1007/s10618-016-0483-9Yemini, E., Jucikas, T., Grundy, L. J., Brown, A. E. X., & Schafer, W. R. (2013). A database of Caenorhabditis elegans behavioral phenotypes. Nature Methods, 10(9), 877-879. doi:10.1038/nmeth.2560Brown, A. E. X., Yemini, E. I., Grundy, L. J., Jucikas, T., & Schafer, W. R. (2012). A dictionary of behavioral motifs reveals clusters of genes affecting Caenorhabditis elegans locomotion. Proceedings of the National Academy of Sciences, 110(2), 791-796. doi:10.1073/pnas.1211447110Cuesta-Frau, D., Novák, D., Burda, V., Molina-Picó, A., Vargas, B., Mraz, M., … Haluzik, M. (2018). Characterization of Artifact Influence on the Classification of Glucose Time Series Using Sample Entropy Statistics. Entropy, 20(11), 871. doi:10.3390/e20110871Bürkner, P.-C., Doebler, P., & Holling, H. (2016). Optimal design of the Wilcoxon-Mann-Whitney-test. Biometrical Journal, 59(1), 25-40. doi:10.1002/bimj.201600022Bian, Z., Ouyang, G., Li, Z., Li, Q., Wang, L., & Li, X. (2016). Weighted-Permutation Entropy Analysis of Resting State EEG from Diabetics with Amnestic Mild Cognitive Impairment. Entropy, 18(8), 307. doi:10.3390/e18080307Deng, B., Liang, L., Li, S., Wang, R., Yu, H., Wang, J., & Wei, X. (2015). Complexity extraction of electroencephalograms in Alzheimer’s disease with weighted-permutation entropy. Chaos: An Interdisciplinary Journal of Nonlinear Science, 25(4), 043105. doi:10.1063/1.491701

Implementasi deteksi serangan epilepsi dari data rekaman EEG menggunakan Weighted Permutation Entropy dan Support Vector Machine

Epilepsi merupakan gangguan neurologis jangka panjang yang ditandai dengan serangan-serangan epileptik. Serangan epileptik dapat terjadi dalam waktu singkat hingga guncangan kuat dalam waktu yang lama. Epilepsi adalah penyakit yang cenderung terjadi secara berulang dan tidak dapat disembuhkan, namun serangan-serangan epileptik yang terjadi dapat dikontrol melalui pengobatan. Pada tugas akhir ini, data rekaman electroencephalogram (EEG) dibagi menjadi beberapa window menggunakan segmentasi atau dekomposisi. Proses selanjutnya adalah mengekstraksi setiap window dengan menggunakan Weighted Permutation Entropy yang menghasilkan satu fitur setiap window. Uji coba fitur menggunakan k-fold cross-validation dengan membagi data menjadi data training dan data testing. Selanjutnya data diklasifikasi menggunakan Support Vector Machine. Data rekaman EEG yang digunakan untuk pengujian ini berasal dari ''Klinik für Epileptologie, Universität Bonn” yang diperoleh secara online yang berjumlah 500 data. Data ini terdiri dari serangan epilepsi (set S) dan bukan serangan epilepsi (set Z, N, O, F) yang masing-masing set terdiri dari 100 data. Set Z direkam dari lima orang sehat dengan mata tertutup dan set O direkam dari lima orang sehat mata terbuka. Set F direkam dari penderita epilepsi yang tidak mengalami serangan di hippocampal formation, set N direkam dari penderita epilepsi yang tidak mengalami sernagan di epileptogenic zone, dan set S direkam dari penderita epilepsi ketika terjadi serangan di epileptogenic zone. Uji coba dilakukan pada data set S digabung dengan setiap set lain. Sehingga data yang digunakan sebanyak 200 data rekaman EEG untuk setiap uji coba. Berdasarkan uji coba, metode tersebut menghasilkan akurasi rata-rata sebesar 91,88%. ================================================================= Epilepsy is a long-term neurological disorder characterized by epileptic seizures. Epileptic seizures can occur in a short period of time until a strong shock for a long time. Epilepsy is a disease that tends to occur repeatedly and cannot be healed, but epileptic seizures that occur can be controlled through treatment. In this undergraduate thesis, the electroencephalogram (EEG) record data will be divided into several windows using segmentation or decomposition. The next process is to extract each window by using a Weighted Permutation Entropy that produces a feature of each window. The feature will be tested using k-fold cross-validation by dividing data into training and testing data. Furthermore data is classified using Support Vector Machine. The EEG record data used for testing in this experiment was taken from the online data collected by 500 online '' Clinical für Epileptologie, Universität Bonn '. This data consists of epileptic seizure (set S) and seizure free (set Z, N, O, F) each of which consists of 100 data. Set Z was recorded from five healthy people when eyes are closed and set O recorded from five healthy people when eyes are opened. Set F was recorded from five epilepsy patients during seizure free in hippocampal formation, N sets was recorded from five epilepsy patients during seizure free in epileptogenic zone, and set S was recorded from five epilepsy patients during seizure in epileptogenic zone. The trial test use set S combined with every other set. So the data used are 200 EEG record data for each test. Based on the trials, the proposed method above gave an average accuracy of 91.88%

Weighted-Permutation Entropy Analysis of Resting State EEG from Diabetics with Amnestic Mild Cognitive Impairment

Diabetes is a significant public health issue as it increases the risk for dementia and Alzheimer’s disease (AD). In this study, we aim to investigate whether weighted-permutation entropy (WPE) and permutation entropy (PE) of resting-state EEG (rsEEG) could be applied as potential objective biomarkers to distinguish type 2 diabetes patients with amnestic mild cognitive impairment (aMCI) from those with normal cognitive function. rsEEG series were acquired from 28 patients with type 2 diabetes (16 aMCI patients and 12 controls), and neuropsychological assessments were performed. The rsEEG signals were analysed using WPE and PE methods. The correlations between the PE or WPE of the rsEEG and the neuropsychological assessments were analysed as well. The WPE in the right temporal (RT) region of the aMCI diabetics was lower than the controls, and the WPE was significantly positively correlated to the scores of the Auditory Verbal Learning Test (AVLT) (AVLT-Immediate recall, AVLT-Delayed recall, AVLT-Delayed recognition) and the Wechsler Adult Intelligence Scale Digit Span Test (WAIS-DST). These findings were not obtained with PE. We concluded that the WPE of rsEEG recordings could distinguish aMCI diabetics from normal cognitive function diabetic controls among the current sample of diabetic patients. Thus, the WPE could be a potential index for assisting diagnosis of aMCI in type 2 diabetes

The Retina in Health and Disease

Vision is the most important sense in higher mammals. The retina is the first step in visual processing and the window to the brain. It is not surprising that problems arising in the retina lead to moderate to severe visual impairments. We offer here a collection of reviews as well as original papers dealing with various aspects of retinal function as well as dysfunction. New approaches in retinal research are described, such as the expression and localization of the endocannabinoid system in the normal retina and the role of cannabinoid receptors that could offer new avenues of research in the development of potential treatments for retinal diseases. Moreover, new insights are offered in advancing knowledge towards the prevention and cure of visual pathologies, mainly AMD, RP, and diabetic retinopathy

Improving somatic health for outpatients with severe mental illness: the development of an intervention

Objective: Patients with severe mental illness (SMI) suffer from more somatic illness than the general population. Possible causes are side effects of neuropsychiatric medication, genetic vulnerability, insufficient health care and lifestyle. This co-morbidity is potentially reversible and augments the costs for health care and diminishes quality of life. Screening on symptoms and risks of somatic diseases and coordination of care are proposed to improve SMI-patients' somatic health status. Methods: A clinical facility was started to improve the somatic health status of patients in an outpatient centre in southern Netherlands. This outpatient centre was added to the specialized care for severe and enduring SMI. The intervention consisted of the inventarisation of side-effects and the detection of gaps in health care provision for 72 patients. This was based on interviewing the patients, laboratory screening, collecting information from their general practitioner and pharmacy. A list was compiled of possible diagnosis and health risks, and a plan of action was made for the treatment. Healthcare consumption, quality of life and general functioning were assessed to analyze cost-effectiveness. Evaluations were performed with the psychiatric care team on the process. Results: Mean annual cost of GP's and medical specialist's consultations were E492. There existed a negative relation between EQ5D VAS and the number of self reported chronic diseases. Conclusion: The authors conclude that the procedure is well feasible, but should be set up in close collaboration with all health care professionals of these patients to make tailor made solutions possible

Epidemiology of Injury in English Women's Super league Football: A Cohort Study

INTRODUCTION: The epidemiology of injury in male professional football has been well documented (Ekstrand, Hägglund, & Waldén, 2011) and used as a basis to understand injury trends for a number of years. The prevalence and incidence of injuries occurring in womens super league football is unknown. The aim of this study is to estimate the prevalence and incidence of injury in an English Super League Women’s Football squad. METHODS: Following ethical approval from Leeds Beckett University, players (n = 25) signed to a Women’s Super League Football club provided written informed consent to complete a self-administered injury survey. Measures of exposure, injury and performance over a 12-month period was gathered. Participants were classified as injured if they reported a football injury that required medical attention or withdrawal from participation for one day or more. Injuries were categorised as either traumatic or overuse and whether the injury was a new injury and/or re-injury of the same anatomical site RESULTS: 43 injuries, including re-injury were reported by the 25 participants providing a clinical incidence of 1.72 injuries per player. Total incidence of injury was 10.8/1000 h (95% CI: 7.5 to 14.03). Participants were at higher risk of injury during a match compared with training (32.4 (95% CI: 15.6 to 48.4) vs 8.0 (95% CI: 5.0 to 10.85)/1000 hours, p 28 days) of which there were three non-contact anterior cruciate ligament (ACL) injuries. The epidemiological incidence proportion was 0.80 (95% CI: 0.64 to 0.95) and the average probability that any player on this team will sustain at least one injury was 80.0% (95% CI: 64.3% to 95.6%) CONCLUSION: This is the first report capturing exposure and injury incidence by anatomical site from a cohort of English players and is comparable to that found in Europe (6.3/1000 h (95% CI 5.4 to 7.36) Larruskain et al 2017). The number of ACL injuries highlights a potential injury burden for a squad of this size. Multi-site prospective investigations into the incidence and prevalence of injury in women’s football are require