Characterizations of generalized entropy functions by functional equations

We shall show that a two-parameter extended entropy function is characterized by a functional equation. As a corollary of this result, we obtain that the Tsallis entropy function is characterized by a functional equation, which is a different form used in \cite{ST} i.e., in Proposition \ref{prop01} in the present paper. We also give an interpretation of the functional equation giving the Tsallis entropy function, in the relation with two non-additive properties

On R\'enyi and Tsallis entropies and divergences for exponential families

Many common probability distributions in statistics like the Gaussian, multinomial, Beta or Gamma distributions can be studied under the unified framework of exponential families. In this paper, we prove that both R\'enyi and Tsallis divergences of distributions belonging to the same exponential family admit a generic closed form expression. Furthermore, we show that R\'enyi and Tsallis entropies can also be calculated in closed-form for sub-families including the Gaussian or exponential distributions, among others.Comment: 7 page

Use of the geometric mean as a statistic for the scale of the coupled Gaussian distributions

The geometric mean is shown to be an appropriate statistic for the scale of a heavy-tailed coupled Gaussian distribution or equivalently the Student's t distribution. The coupled Gaussian is a member of a family of distributions parameterized by the nonlinear statistical coupling which is the reciprocal of the degree of freedom and is proportional to fluctuations in the inverse scale of the Gaussian. Existing estimators of the scale of the coupled Gaussian have relied on estimates of the full distribution, and they suffer from problems related to outliers in heavy-tailed distributions. In this paper, the scale of a coupled Gaussian is proven to be equal to the product of the generalized mean and the square root of the coupling. From our numerical computations of the scales of coupled Gaussians using the generalized mean of random samples, it is indicated that only samples from a Cauchy distribution (with coupling parameter one) form an unbiased estimate with diminishing variance for large samples. Nevertheless, we also prove that the scale is a function of the geometric mean, the coupling term and a harmonic number. Numerical experiments show that this estimator is unbiased with diminishing variance for large samples for a broad range of coupling values.Comment: 17 pages, 5 figure

The Statistical Foundations of Entropy

In the last two decades, the understanding of complex dynamical systems underwent important conceptual shifts. The catalyst was the infusion of new ideas from the theory of critical phenomena (scaling laws, renormalization group, etc.), (multi)fractals and trees, random matrix theory, network theory, and non-Shannonian information theory. The usual Boltzmann–Gibbs statistics were proven to be grossly inadequate in this context. While successful in describing stationary systems characterized by ergodicity or metric transitivity, Boltzmann–Gibbs statistics fail to reproduce the complex statistical behavior of many real-world systems in biology, astrophysics, geology, and the economic and social sciences.The aim of this Special Issue was to extend the state of the art by original contributions that could contribute to an ongoing discussion on the statistical foundations of entropy, with a particular emphasis on non-conventional entropies that go significantly beyond Boltzmann, Gibbs, and Shannon paradigms. The accepted contributions addressed various aspects including information theoretic, thermodynamic and quantum aspects of complex systems and found several important applications of generalized entropies in various systems

Extended Entropy Maximisation and Queueing Systems with Heavy-Tailed Distributions

Numerous studies on Queueing systems, such as Internet traffic flows, have shown to be bursty, self-similar and/or long-range dependent, because of the heavy (long) tails for the various distributions of interest, including intermittent intervals and queue lengths. Other studies have addressed vacation in no-customers’ queueing system or when the server fails. These patterns are important for capacity planning, performance prediction, and optimization of networks and have a negative impact on their effective functioning. Heavy-tailed distributions have been commonly used by telecommunication engineers to create workloads for simulation studies, which, regrettably, may show peculiar queueing characteristics. To cost-effectively examine the impacts of different network patterns on heavy- tailed queues, new and reliable analytical approaches need to be developed. It is decided to establish a brand-new analytical framework based on optimizing entropy functionals, such as those of Shannon, Rényi, Tsallis, and others that have been suggested within statistical physics and information theory, subject to suitable linear and non-linear system constraints. In both discrete and continuous time domains, new heavy tail analytic performance distributions will be developed, with a focus on those exhibiting the power law behaviour seen in many Internet scenarios. The exposition of two major revolutionary approaches, namely the unification of information geometry and classical queueing systems and unifying information length theory with transient queueing systems. After conclusions, open problems arising from this thesis and limitations are introduced as future work

Entropies from coarse-graining: convex polytopes vs. ellipsoids

We examine the Boltzmann/Gibbs/Shannon $\mathcal{S}_{BGS}$ and the non-additive Havrda-Charv\'{a}t / Dar\'{o}czy/Cressie-Read/Tsallis \ $\mathcal{S}_q$ \ and the Kaniadakis $\kappa$-entropy \ $\mathcal{S}_\kappa$ \ from the viewpoint of coarse-graining, symplectic capacities and convexity. We argue that the functional form of such entropies can be ascribed to a discordance in phase-space coarse-graining between two generally different approaches: the Euclidean/Riemannian metric one that reflects independence and picks cubes as the fundamental cells and the symplectic/canonical one that picks spheres/ellipsoids for this role. Our discussion is motivated by and confined to the behaviour of Hamiltonian systems of many degrees of freedom. We see that Dvoretzky's theorem provides asymptotic estimates for the minimal dimension beyond which these two approaches are close to each other. We state and speculate about the role that dualities may play in this viewpoint.Comment: 63 pages. No figures. Standard LaTe