Entropy and Long range correlations in literary English

Recently long range correlations were detected in nucleotide sequences and in human writings by several authors. We undertake here a systematic investigation of two books, Moby Dick by H. Melville and Grimm's tales, with respect to the existence of long range correlations. The analysis is based on the calculation of entropy like quantities as the mutual information for pairs of letters and the entropy, the mean uncertainty, per letter. We further estimate the number of different subwords of a given length $n$. Filtering out the contributions due to the effects of the finite length of the texts, we find correlations ranging to a few hundred letters. Scaling laws for the mutual information (decay with a power law), for the entropy per letter (decay with the inverse square root of $n$) and for the word numbers (stretched exponential growth with $n$ and with a power law of the text length) were found.Comment: 8 page

Spreading and shortest paths in systems with sparse long-range connections

Spreading according to simple rules (e.g. of fire or diseases), and shortest-path distances are studied on d-dimensional systems with a small density p per site of long-range connections (``Small-World'' lattices). The volume V(t) covered by the spreading quantity on an infinite system is exactly calculated in all dimensions. We find that V(t) grows initially as t^d/d for t>t^*$, generalizing a previous result in one dimension. Using the properties of V(t), the average shortest-path distance \ell(r) can be calculated as a function of Euclidean distance r. It is found that \ell(r) = r for r<r_c=(2p \Gamma_d (d-1)!)^{-1/d} log(2p \Gamma_d L^d), and \ell(r) = r_c for r>r_c. The characteristic length r_c, which governs the behavior of shortest-path lengths, diverges with system size for all p>0. Therefore the mean separation s \sim p^{-1/d} between shortcut-ends is not a relevant internal length-scale for shortest-path lengths. We notice however that the globally averaged shortest-path length, divided by L, is a function of L/s only.Comment: 4 pages, 1 eps fig. Uses psfi

Finite-sample frequency distributions originating from an equiprobability distribution

Given an equidistribution for probabilities p(i)=1/N, i=1..N. What is the expected corresponding rank ordered frequency distribution f(i), i=1..N, if an ensemble of M events is drawn?Comment: 4 pages, 4 figure

Guessing probability distributions from small samples

We propose a new method for the calculation of the statistical properties, as e.g. the entropy, of unknown generators of symbolic sequences. The probability distribution $p(k)$ of the elements $k$ of a population can be approximated by the frequencies $f(k)$ of a sample provided the sample is long enough so that each element $k$ occurs many times. Our method yields an approximation if this precondition does not hold. For a given $f(k)$ we recalculate the Zipf--ordered probability distribution by optimization of the parameters of a guessed distribution. We demonstrate that our method yields reliable results.Comment: 10 pages, uuencoded compressed PostScrip

Statistics of finite-time Lyapunov exponents in the Ulam map

The statistical properties of finite-time Lyapunov exponents at the Ulam point of the logistic map are investigated. The exact analytical expression for the autocorrelation function of one-step Lyapunov exponents is obtained, allowing the calculation of the variance of exponents computed over time intervals of length $n$. The variance anomalously decays as $1/n^2$. The probability density of finite-time exponents noticeably deviates from the Gaussian shape, decaying with exponential tails and presenting $2^{n-1}$ spikes that narrow and accumulate close to the mean value with increasing $n$. The asymptotic expression for this probability distribution function is derived. It provides an adequate smooth approximation to describe numerical histograms built for not too small $n$, where the finiteness of bin size trimmes the sharp peaks.Comment: 6 pages, 4 figures, to appear in Phys. Rev.

On the relationship between directed percolation and the synchronization transition in spatially extended systems

We study the nature of the synchronization transition in spatially extended systems by discussing a simple stochastic model. An analytic argument is put forward showing that, in the limit of discontinuous processes, the transition belongs to the directed percolation (DP) universality class. The analysis is complemented by a detailed investigation of the dependence of the first passage time for the amplitude of the difference field on the adopted threshold. We find the existence of a critical threshold separating the regime controlled by linear mechanisms from that controlled by collective phenomena. As a result of this analysis we conclude that the synchronization transition belongs to the DP class also in continuous models. The conclusions are supported by numerical checks on coupled map lattices too

Entropy estimates of small data sets

Estimating entropies from limited data series is known to be a non-trivial task. Naive estimations are plagued with both systematic (bias) and statistical errors. Here, we present a new 'balanced estimator' for entropy functionals Shannon, R\'enyi and Tsallis) specially devised to provide a compromise between low bias and small statistical errors, for short data series. This new estimator out-performs other currently available ones when the data sets are small and the probabilities of the possible outputs of the random variable are not close to zero. Otherwise, other well-known estimators remain a better choice. The potential range of applicability of this estimator is quite broad specially for biological and digital data series.Comment: 11 pages, 2 figure