27,916 research outputs found
Renormalization Group and Quantum Information
The renormalization group is a tool that allows one to obtain a reduced
description of systems with many degrees of freedom while preserving the
relevant features. In the case of quantum systems, in particular,
one-dimensional systems defined on a chain, an optimal formulation is given by
White's "density matrix renormalization group". This formulation can be shown
to rely on concepts of the developing theory of quantum information.
Furthermore, White's algorithm can be connected with a peculiar type of
quantization, namely, angular quantization. This type of quantization arose in
connection with quantum gravity problems, in particular, the Unruh effect in
the problem of black-hole entropy and Hawking radiation. This connection
highlights the importance of quantum system boundaries, regarding the
concentration of quantum states on them, and helps us to understand the optimal
nature of White's algorithm.Comment: 16 pages, 5 figures, accepted in Journal of Physics
Likelihood Analysis of Power Spectra and Generalized Moment Problems
We develop an approach to spectral estimation that has been advocated by
Ferrante, Masiero and Pavon and, in the context of the scalar-valued covariance
extension problem, by Enqvist and Karlsson. The aim is to determine the power
spectrum that is consistent with given moments and minimizes the relative
entropy between the probability law of the underlying Gaussian stochastic
process to that of a prior. The approach is analogous to the framework of
earlier work by Byrnes, Georgiou and Lindquist and can also be viewed as a
generalization of the classical work by Burg and Jaynes on the maximum entropy
method. In the present paper we present a new fast algorithm in the general
case (i.e., for general Gaussian priors) and show that for priors with a
specific structure the solution can be given in closed form.Comment: 17 pages, 4 figure
A simple probabilistic construction yielding generalized entropies and divergences, escort distributions and q-Gaussians
We give a simple probabilistic description of a transition between two states
which leads to a generalized escort distribution. When the parameter of the
distribution varies, it defines a parametric curve that we call an escort-path.
The R\'enyi divergence appears as a natural by-product of the setting. We study
the dynamics of the Fisher information on this path, and show in particular
that the thermodynamic divergence is proportional to Jeffreys' divergence.
Next, we consider the problem of inferring a distribution on the escort-path,
subject to generalized moments constraints. We show that our setting naturally
induces a rationale for the minimization of the R\'enyi information divergence.
Then, we derive the optimum distribution as a generalized q-Gaussian
distribution
On Some Integrated Approaches to Inference
We present arguments for the formulation of unified approach to different
standard continuous inference methods from partial information. It is claimed
that an explicit partition of information into a priori (prior knowledge) and a
posteriori information (data) is an important way of standardizing inference
approaches so that they can be compared on a normative scale, and so that
notions of optimal algorithms become farther-reaching. The inference methods
considered include neural network approaches, information-based complexity, and
Monte Carlo, spline, and regularization methods. The model is an extension of
currently used continuous complexity models, with a class of algorithms in the
form of optimization methods, in which an optimization functional (involving
the data) is minimized. This extends the family of current approaches in
continuous complexity theory, which include the use of interpolatory algorithms
in worst and average case settings
Time and spectral domain relative entropy: A new approach to multivariate spectral estimation
The concept of spectral relative entropy rate is introduced for jointly
stationary Gaussian processes. Using classical information-theoretic results,
we establish a remarkable connection between time and spectral domain relative
entropy rates. This naturally leads to a new spectral estimation technique
where a multivariate version of the Itakura-Saito distance is employed}. It may
be viewed as an extension of the approach, called THREE, introduced by Byrnes,
Georgiou and Lindquist in 2000 which, in turn, followed in the footsteps of the
Burg-Jaynes Maximum Entropy Method. Spectral estimation is here recast in the
form of a constrained spectrum approximation problem where the distance is
equal to the processes relative entropy rate. The corresponding solution
entails a complexity upper bound which improves on the one so far available in
the multichannel framework. Indeed, it is equal to the one featured by THREE in
the scalar case. The solution is computed via a globally convergent matricial
Newton-type algorithm. Simulations suggest the effectiveness of the new
technique in tackling multivariate spectral estimation tasks, especially in the
case of short data records.Comment: 32 pages, submitted for publicatio
Information Theory - The Bridge Connecting Bounded Rational Game Theory and Statistical Physics
A long-running difficulty with conventional game theory has been how to
modify it to accommodate the bounded rationality of all real-world players. A
recurring issue in statistical physics is how best to approximate joint
probability distributions with decoupled (and therefore far more tractable)
distributions. This paper shows that the same information theoretic
mathematical structure, known as Product Distribution (PD) theory, addresses
both issues. In this, PD theory not only provides a principled formulation of
bounded rationality and a set of new types of mean field theory in statistical
physics. It also shows that those topics are fundamentally one and the same.Comment: 17 pages, no figures, accepted for publicatio
- âŠ