10 research outputs found
Nonextensive triangle equality and other properties of Tsallis relative-entropy minimization
Kullback-Leibler relative-entropy has unique properties in cases involving
distributions resulting from relative-entropy minimization. Tsallis
relative-entropy is a one parameter generalization of Kullback-Leibler
relative-entropy in the nonextensive thermostatistics. In this paper, we
present the properties of Tsallis relative-entropy minimization and present
some differences with the classical case. In the representation of such a
minimum relative-entropy distribution, we highlight the use of the q-product,
an operator that has been recently introduced to derive the mathematical
structure behind the Tsallis statistics. One of our main results is
generalization of triangle equality of relative-entropy minimization to the
nonextensive case.Comment: 15 pages, change of title, revision of triangle equalit
On uniqueness theorems for Tsallis entropy and Tsallis relative entropy
The uniqueness theorem for Tsallis entropy was presented in {\it H.Suyari,
IEEE Trans. Inform. Theory, Vol.50, pp.1783-1787 (2004)} by introducing the
generalized Shannon-Khinchin's axiom. In the present paper, this result is
generalized and simplified as follows: {\it Generalization}: The uniqueness
theorem for Tsallis relative entropy is shown by means of the generalized
Hobson's axiom. {\it Simplification}: The uniqueness theorem for Tsallis
entropy is shown by means of the generalized Faddeev's axiom.Comment: this was merged by two manuscripts (arXiv:cond-mat/0410270 and
arXiv:cond-mat/0410271), and will be published from IEEE TI
Information theoretical properties of Tsallis entropies
A chain rule and a subadditivity for the entropy of type , which is
one of the nonadditive entropies, were derived by Z.Dar\'oczy. In this paper,
we study the further relations among Tsallis type entropies which are typical
nonadditive entropies. The chain rule is generalized by showing it for Tsallis
relative entropy and the nonadditive entropy. We show some inequalities related
to Tsallis entropies, especially the strong subadditivity for Tsallis type
entropies and the subadditivity for the nonadditive entropies. The
subadditivity and the strong subadditivity naturally lead to define Tsallis
mutual entropy and Tsallis conditional mutual entropy, respectively, and then
we show again chain rules for Tsallis mutual entropies. We give properties of
entropic distances in terms of Tsallis entropies. Finally we show
parametrically extended results based on information theory.Comment: The subsection on data processing inequality was deleted. Some typo's
were modifie
Thermodynamic semirings
The Witt construction describes a functor from the category of Rings to the category
of characteristic 0 rings. It is uniquely determined by a few associativity constraints which do
not depend on the types of the variables considered, in other words, by integer polynomials.
This universality allowed Alain Connes and Caterina Consani to devise an analogue of the Witt
ring for characteristic one, an attractive endeavour since we know very little about the arithmetic
in this exotic characteristic and its corresponding field with one element. Interestingly, they
found that in characteristic one, the Witt construction depends critically on the Shannon entropy.
In the current work, we examine this surprising occurrence, defining a Witt operad for an
arbitrary information measure and a corresponding algebra we call a thermodynamic semiring.
This object exhibits algebraically many of the familiar properties of information measures,
and we examine in particular the Tsallis and Renyi entropy functions and applications to nonextensive
thermodynamics and multifractals. We find that the arithmetic of the thermodynamic
semiring is exactly that of a certain guessing game played using the given information measure
Contributos para a teoria de máxima entropia na estimação de modelos mal-postos
Doutoramento em MatemáticaAs técnicas estatísticas são fundamentais em ciência e a análise de regressão
linear é, quiçá, uma das metodologias mais usadas. É bem conhecido da literatura
que, sob determinadas condições, a regressão linear é uma ferramenta
estatística poderosíssima. Infelizmente, na prática, algumas dessas condições
raramente são satisfeitas e os modelos de regressão tornam-se mal-postos,
inviabilizando, assim, a aplicação dos tradicionais métodos de estimação.
Este trabalho apresenta algumas contribuições para a teoria de máxima entropia
na estimação de modelos mal-postos, em particular na estimação de modelos
de regressão linear com pequenas amostras, afetados por colinearidade
e outliers. A investigação é desenvolvida em três vertentes, nomeadamente na
estimação de eficiência técnica com fronteiras de produção condicionadas a
estados contingentes, na estimação do parâmetro ridge em regressão ridge e,
por último, em novos desenvolvimentos na estimação com máxima entropia.
Na estimação de eficiência técnica com fronteiras de produção condicionadas
a estados contingentes, o trabalho desenvolvido evidencia um melhor desempenho
dos estimadores de máxima entropia em relação ao estimador de máxima
verosimilhança. Este bom desempenho é notório em modelos com poucas
observações por estado e em modelos com um grande número de estados,
os quais são comummente afetados por colinearidade. Espera-se que a
utilização de estimadores de máxima entropia contribua para o tão desejado
aumento de trabalho empírico com estas fronteiras de produção.
Em regressão ridge o maior desafio é a estimação do parâmetro ridge. Embora
existam inúmeros procedimentos disponíveis na literatura, a verdade é que
não existe nenhum que supere todos os outros. Neste trabalho é proposto um
novo estimador do parâmetro ridge, que combina a análise do traço ridge e a
estimação com máxima entropia. Os resultados obtidos nos estudos de simulação
sugerem que este novo estimador é um dos melhores procedimentos
existentes na literatura para a estimação do parâmetro ridge.
O estimador de máxima entropia de Leuven é baseado no método dos mínimos
quadrados, na entropia de Shannon e em conceitos da eletrodinâmica
quântica. Este estimador suplanta a principal crítica apontada ao estimador de
máxima entropia generalizada, uma vez que prescinde dos suportes para os
parâmetros e erros do modelo de regressão. Neste trabalho são apresentadas
novas contribuições para a teoria de máxima entropia na estimação de modelos
mal-postos, tendo por base o estimador de máxima entropia de Leuven, a
teoria da informação e a regressão robusta. Os estimadores desenvolvidos
revelam um bom desempenho em modelos de regressão linear com pequenas
amostras, afetados por colinearidade e outliers.
Por último, são apresentados alguns códigos computacionais para estimação
com máxima entropia, contribuindo, deste modo, para um aumento dos escassos
recursos computacionais atualmente disponíveis.Statistical techniques are essential in most areas of science being linear regression
one of the most widely used. It is well-known that under fairly conditions
linear regression is a powerful statistical tool. Unfortunately, some of
these conditions are usually not satisfied in practice and the regression models
become ill-posed, which means that the application of traditional estimation
methods may lead to non-unique or highly unstable solutions.
This work is mainly focused on the maximum entropy estimation of ill-posed
models, in particular the estimation of regression models with small samples
sizes affected by collinearity and outliers. The research is developed in three
directions, namely the estimation of technical efficiency with state-contingent
production frontiers, the estimation of the ridge parameter in ridge regression,
and some developments in maximum entropy estimation.
In the estimation of technical efficiency with state-contingent production frontiers,
this work reveals that the maximum entropy estimators outperform the
maximum likelihood estimator in most of the cases analyzed, namely in models
with few observations in some states of nature and models with a large number
of states of nature, which usually represent models affected by collinearity. The
maximum entropy estimators are expected to make an important contribution to
the increase of empirical work with state-contingent production frontiers.
The main challenge in ridge regression is the selection of the ridge parameter.
There is a huge number of methods to estimate the ridge parameter and no
single method emerges in the literature as the best overall. In this work, a new
method to select the ridge parameter in ridge regression is presented. The
simulation study reveals that, in the case of regression models with small samples
sizes affected by collinearity, the new estimator is probably one of the best
ridge parameter estimators available in the literature on ridge regression.
Founded on the Shannon entropy, the ordinary least squares estimator and
some concepts from quantum electrodynamics, the maximum entropy Leuven
estimator overcomes the main weakness of the generalized maximum entropy
estimator, avoiding exogenous information that is usually not available. Based
on the maximum entropy Leuven estimator, information theory and robust regression,
new developments on the theory of maximum entropy estimation are
provided in this work. The simulation studies and the empirical applications reveal
that the new estimators are a good choice in the estimation of linear regression
models with small samples sizes affected by collinearity and outliers.
Finally, a contribution to the increase of computational resources on the maximum
entropy estimation is also accomplished in this work