On some interrelations of generalized qq-entropies and a generalized Fisher information, including a Cram\'er-Rao inequality

In this communication, we describe some interrelations between generalized $q$-entropies and a generalized version of Fisher information. In information theory, the de Bruijn identity links the Fisher information and the derivative of the entropy. We show that this identity can be extended to generalized versions of entropy and Fisher information. More precisely, a generalized Fisher information naturally pops up in the expression of the derivative of the Tsallis entropy. This generalized Fisher information also appears as a special case of a generalized Fisher information for estimation problems. Indeed, we derive here a new Cram\'er-Rao inequality for the estimation of a parameter, which involves a generalized form of Fisher information. This generalized Fisher information reduces to the standard Fisher information as a particular case. In the case of a translation parameter, the general Cram\'er-Rao inequality leads to an inequality for distributions which is saturated by generalized $q$-Gaussian distributions. These generalized $q$-Gaussians are important in several areas of physics and mathematics. They are known to maximize the $q$-entropies subject to a moment constraint. The Cram\'er-Rao inequality shows that the generalized $q$-Gaussians also minimize the generalized Fisher information among distributions with a fixed moment. Similarly, the generalized $q$-Gaussians also minimize the generalized Fisher information among distributions with a given $q$-entropy

Source coding with escort distributions and Renyi entropy bounds

We discuss the interest of escort distributions and R\'enyi entropy in the context of source coding. We first recall a source coding theorem by Campbell relating a generalized measure of length to the R\'enyi-Tsallis entropy. We show that the associated optimal codes can be obtained using considerations on escort-distributions. We propose a new family of measure of length involving escort-distributions and we show that these generalized lengths are also bounded below by the R\'enyi entropy. Furthermore, we obtain that the standard Shannon codes lengths are optimum for the new generalized lengths measures, whatever the entropic index. Finally, we show that there exists in this setting an interplay between standard and escort distributions

An amended MaxEnt formulation for deriving Tsallis factors, and associated issues

An amended MaxEnt formulation for systems displaced from the conventional MaxEnt equilibrium is proposed. This formulation involves the minimization of the Kullback-Leibler divergence to a reference $Q$ (or maximization of Shannon $Q$-entropy), subject to a constraint that implicates a second reference distribution $P\_{1}$ and tunes the new equilibrium. In this setting, the equilibrium distribution is the generalized escort distribution associated to $P\_{1}$ and $Q$. The account of an additional constraint, an observable given by a statistical mean, leads to the maximization of R\'{e}nyi/Tsallis $Q$-entropy subject to that constraint. Two natural scenarii for this observation constraint are considered, and the classical and generalized constraint of nonextensive statistics are recovered. The solutions to the maximization of R\'{e}nyi $Q$-entropy subject to the two types of constraints are derived. These optimum distributions, that are Levy-like distributions, are self-referential. We then propose two `alternate' (but effectively computable) dual functions, whose maximizations enable to identify the optimum parameters. Finally, a duality between solutions and the underlying Legendre structure are presented.Comment: Presented at MaxEnt2006, Paris, France, july 10-13, 200

A simple probabilistic construction yielding generalized entropies and divergences, escort distributions and q-Gaussians

We give a simple probabilistic description of a transition between two states which leads to a generalized escort distribution. When the parameter of the distribution varies, it defines a parametric curve that we call an escort-path. The R\'enyi divergence appears as a natural by-product of the setting. We study the dynamics of the Fisher information on this path, and show in particular that the thermodynamic divergence is proportional to Jeffreys' divergence. Next, we consider the problem of inferring a distribution on the escort-path, subject to generalized moments constraints. We show that our setting naturally induces a rationale for the minimization of the R\'enyi information divergence. Then, we derive the optimum distribution as a generalized q-Gaussian distribution

Quelques inégalités caractérisant les gaussiennes généralisées

International audienceDans cet article, on propose de caractériser les gaussiennes généralisées comme les densités atteignant les bornes de plusieurs extensions d'inégalités connues en théorie de l'information. On retrouve en cas particulier les résultats pour la gaussienne standard. Abstract - In this paper, we propose to characterize a class of generalized Gaussian distributions as the densities that saturate several extensions of classical information theoretic inequalities. The results for the standard Gaussian are recovered as a particular case

On escort distributions, q-gaussians and Fisher information

International audienceEscort distributions are a simple one parameter deformation of an original distribution p. In Tsallis extended thermostatistics, the escort‐averages, defined with respect to an escort distribution, have revealed useful in order to obtain analytical results and variational equations, with in particular the equilibrium distributions obtained as maxima of Rényi‐Tsallis entropy subject to constraints in the form of a q‐average. A central example is the q‐gaussian, which is a generalization of the standard gaussian distribution. In this contribution, we show that escort distributions emerge naturally as a maximum entropy trade‐off between the distribution p(x) and the uniform distribution. This setting may typically describe a phase transition between two states. But escort distributions also appear in the fields of multifractal analysis, quantization and coding with interesting consequences. For the problem of coding, we recall a source coding theorem by Campbell relating a generalized measure of length to the Rényi‐Tsallis entropy and exhibit the links with escort distributions together with pratical implications. That q‐gaussians arise from the maximization of Rényi‐Tsallis entropy subject to a q‐variance constraint is a known fact. We show here that the (squared) q‐gaussian also appear as a minimum of Fisher information subject to the same q‐variance constraint

Escort entropies and divergences and related canonical distribution

arXiv : 1109.3311International audienceWe discuss two families of two-parameter entropies and divergences, derived from the standard Rényi and Tsallis entropies and divergences. These divergences and entropies are found as divergences or entropies of escort distributions. Exploiting the nonnegativity of the divergences, we derive the expression of the canonical distribution associated to the new entropies and a observable given as an escort-mean value. We show that this canonical distribution extends, and smoothly connects, the results obtained in nonextensive thermodynamics for the standard and generalized mean value constraints

Entropies et critères entropiques

International audienceCe chapitre est centré sur les notions d'entropies et de lois à maximum d'entropie qui seront caractérisées selon plusieurs angles. Au delà des liens avec les applications en ingénierie puis en physique, on montrera qu'il est possible de bâtir des fonction-nelles régularisantes fondées sur l'emploi d'une technique à maximum d'entropie, qui peuvent alors éventuellement être utilisées comme potentiels ad hoc dans des pro-blèmes d'inversion de données. Le chapitre débute par un tour d'horizon des principales propriétés des mesures d'information, et par l'introduction de différentes notions et définitions. En particu-lier, on définit la divergence de Rényi, on présente la notion de distribution escorte, et on commente le principe du maximum d'entropie qui sera utilisé par la suite. On présente ensuite un problème classique d'ingénierie, le problème du codage de source, et on montre l'intérêt d'utiliser des mesures de longueur différentes de la mesure stan-dard, et en particulier une mesure exponentielle, qui conduit à un théorème de codage de source dont la borne minimale est une entropie de Rényi. On montre également que les codes optimaux peuvent être calculés aisément grâce aux distributions escortes. En section 1.4, on introduit et on étudie un modèle simple de transition d'état. Ce modèle conduit à une distribution d'équilibre définie comme une distribution escorte généra-lisée, et conduit en sous-produit, à nouveau à une entropie de Rényi. On étudie le flux d'information de Fisher le long de la courbe définie par la distribution escorte géné-ralisée, et on obtient des connections avec la divergence de Jeffreys. Finalement, on obtient différents arguments qui, dans ce cadre, conduisent à une méthode d'inférence fondée sur la minimisation de l'entropie de Rényi sous une contrainte de moyenne généralisée, i.e. prise vis-à-vis de la distribution escorte. À partir de la section 1.5.3, on s'intéresse alors à la minimisation de la divergence de Rényi sous une contraint