Maximum Entropy and Bayesian Data Analysis: Entropic Priors

The problem of assigning probability distributions which objectively reflect the prior information available about experiments is one of the major stumbling blocks in the use of Bayesian methods of data analysis. In this paper the method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is inspired and guided by intuition gained from the successful use of ME methods in statistical mechanics. For experiments that cannot be repeated the resulting "entropic prior" is formally identical with the Einstein fluctuation formula. For repeatable experiments, however, the expected value of the entropy of the likelihood turns out to be relevant information that must be included in the analysis. The important case of a Gaussian likelihood is treated in detail.Comment: 23 pages, 2 figure

On wind Turbine failure detection from measurements of phase currents: a permutation entropy approach

This article presents the applicability of Permutation Entropy based complexity measure of a time series for detection of fault in wind turbines. A set of electrical data from one faulty and one healthy wind turbine were analysed using traditional FastFourier analysis in addition to Permutation Entropy analysis to compare the complexity index of phase currents of the two turbines over time. The 4 seconds length data set did not reveal any low frequency in the spectra of currents, neither did they show any meaningful differences of spectrum between the two turbine currents. Permutation Entropy analysis of the current waveforms of same phases for the two turbines are found to have different complexity values over time, one of them being clearly higher than the other. The work of Yan et. al. in has found that higher entropy values related to thepresence of failure in rotary machines in his study. Following this track, further efforts will be put into relating the entropy difference found in our study to possible presence of failure in one of the wind energy conversion systems

Delay Parameter Selection in Permutation Entropy Using Topological Data Analysis

Permutation Entropy (PE) is a powerful tool for quantifying the predictability of a sequence which includes measuring the regularity of a time series. Despite its successful application in a variety of scientific domains, PE requires a judicious choice of the delay parameter $\tau$. While another parameter of interest in PE is the motif dimension $n$, Typically $n$ is selected between $4$ and $8$ with $5$ or $6$ giving optimal results for the majority of systems. Therefore, in this work we focus solely on choosing the delay parameter. Selecting $\tau$ is often accomplished using trial and error guided by the expertise of domain scientists. However, in this paper, we show that persistent homology, the flag ship tool from Topological Data Analysis (TDA) toolset, provides an approach for the automatic selection of $\tau$. We evaluate the successful identification of a suitable $\tau$ from our TDA-based approach by comparing our results to a variety of examples in published literature

Entropy in the natural time-domain

A surrogate data analysis is presented, which is based on the fluctuations of the ``entropy'' $S$ defined in the natural time-domain [Phys. Rev. E {\bf 68}, 031106, 2003]. This entropy is not a static one as, for example, the Shannon entropy. The analysis is applied to three types of time-series, i.e., seismic electric signals, ``artificial'' noises and electrocardiograms, and ``recognizes'' the non-Markovianity in all these signals. Furthermore, it differentiates the electrocardiograms of healthy humans from those of the sudden cardiac death ones. If $\delta S$ and $\delta S_{shuf}$ denote the standard deviation when calculating the entropy by means of a time-window sweeping through the original data and the ``shuffled'' (randomized) data, respectively, it seems that the ratio $\delta S_{shuf}/\delta S$ plays a key-role. The physical meaning of $\delta S_{shuf}$ is investigated.Comment: Published in Physical Review

Point Information Gain and Multidimensional Data Analysis

We generalize the Point information gain (PIG) and derived quantities, i.e. Point information entropy (PIE) and Point information entropy density (PIED), for the case of R\'enyi entropy and simulate the behavior of PIG for typical distributions. We also use these methods for the analysis of multidimensional datasets. We demonstrate the main properties of PIE/PIED spectra for the real data on the example of several images, and discuss possible further utilization in other fields of data processing.Comment: 16 pages, 6 figure

Anomaly Detection in Paleoclimate Records using Permutation Entropy

Permutation entropy techniques can be useful in identifying anomalies in paleoclimate data records, including noise, outliers, and post-processing issues. We demonstrate this using weighted and unweighted permutation entropy of water-isotope records in a deep polar ice core. In one region of these isotope records, our previous calculations revealed an abrupt change in the complexity of the traces: specifically, in the amount of new information that appeared at every time step. We conjectured that this effect was due to noise introduced by an older laboratory instrument. In this paper, we validate that conjecture by re-analyzing a section of the ice core using a more-advanced version of the laboratory instrument. The anomalous noise levels are absent from the permutation entropy traces of the new data. In other sections of the core, we show that permutation entropy techniques can be used to identify anomalies in the raw data that are not associated with climatic or glaciological processes, but rather effects occurring during field work, laboratory analysis, or data post-processing. These examples make it clear that permutation entropy is a useful forensic tool for identifying sections of data that require targeted re-analysis---and can even be useful in guiding that analysis.Comment: 15 pages, 7 figure