160 research outputs found
Likelihood Analysis of Power Spectra and Generalized Moment Problems
We develop an approach to spectral estimation that has been advocated by
Ferrante, Masiero and Pavon and, in the context of the scalar-valued covariance
extension problem, by Enqvist and Karlsson. The aim is to determine the power
spectrum that is consistent with given moments and minimizes the relative
entropy between the probability law of the underlying Gaussian stochastic
process to that of a prior. The approach is analogous to the framework of
earlier work by Byrnes, Georgiou and Lindquist and can also be viewed as a
generalization of the classical work by Burg and Jaynes on the maximum entropy
method. In the present paper we present a new fast algorithm in the general
case (i.e., for general Gaussian priors) and show that for priors with a
specific structure the solution can be given in closed form.Comment: 17 pages, 4 figure
Time and spectral domain relative entropy: A new approach to multivariate spectral estimation
The concept of spectral relative entropy rate is introduced for jointly
stationary Gaussian processes. Using classical information-theoretic results,
we establish a remarkable connection between time and spectral domain relative
entropy rates. This naturally leads to a new spectral estimation technique
where a multivariate version of the Itakura-Saito distance is employed}. It may
be viewed as an extension of the approach, called THREE, introduced by Byrnes,
Georgiou and Lindquist in 2000 which, in turn, followed in the footsteps of the
Burg-Jaynes Maximum Entropy Method. Spectral estimation is here recast in the
form of a constrained spectrum approximation problem where the distance is
equal to the processes relative entropy rate. The corresponding solution
entails a complexity upper bound which improves on the one so far available in
the multichannel framework. Indeed, it is equal to the one featured by THREE in
the scalar case. The solution is computed via a globally convergent matricial
Newton-type algorithm. Simulations suggest the effectiveness of the new
technique in tackling multivariate spectral estimation tasks, especially in the
case of short data records.Comment: 32 pages, submitted for publicatio
Regularized Optimal Transport and the Rot Mover's Distance
This paper presents a unified framework for smooth convex regularization of
discrete optimal transport problems. In this context, the regularized optimal
transport turns out to be equivalent to a matrix nearness problem with respect
to Bregman divergences. Our framework thus naturally generalizes a previously
proposed regularization based on the Boltzmann-Shannon entropy related to the
Kullback-Leibler divergence, and solved with the Sinkhorn-Knopp algorithm. We
call the regularized optimal transport distance the rot mover's distance in
reference to the classical earth mover's distance. We develop two generic
schemes that we respectively call the alternate scaling algorithm and the
non-negative alternate scaling algorithm, to compute efficiently the
regularized optimal plans depending on whether the domain of the regularizer
lies within the non-negative orthant or not. These schemes are based on
Dykstra's algorithm with alternate Bregman projections, and further exploit the
Newton-Raphson method when applied to separable divergences. We enhance the
separable case with a sparse extension to deal with high data dimensions. We
also instantiate our proposed framework and discuss the inherent specificities
for well-known regularizers and statistical divergences in the machine learning
and information geometry communities. Finally, we demonstrate the merits of our
methods with experiments using synthetic data to illustrate the effect of
different regularizers and penalties on the solutions, as well as real-world
data for a pattern recognition application to audio scene classification
Conal Distances Between Rational Spectral Densities
This paper generalizes Thompson and Hilbert
metrics to the space of spectral densities. The resulting
complete metric space has the differentiable structure of a
Finsler manifold with explicit geodesics. The corresponding distances are filtering invariant, can be computed efficiently, and admit geodesic paths that preserve rationality; these are properties of fundamental importance in many
engineering applications.European Research Counci
Computationally-efficient initialisation of GPs: The generalised variogram method
We present a computationally-efficient strategy to find the hyperparameters
of a Gaussian process (GP) avoiding the computation of the likelihood function.
The found hyperparameters can then be used directly for regression or passed as
initial conditions to maximum-likelihood (ML) training. Motivated by the fact
that training a GP via ML is equivalent (on average) to minimising the
KL-divergence between the true and learnt model, we set to explore different
metrics/divergences among GPs that are computationally inexpensive and provide
estimates close to those of ML. In particular, we identify the GP
hyperparameters by projecting the empirical covariance or (Fourier) power
spectrum onto a parametric family, thus proposing and studying various measures
of discrepancy operating on the temporal or frequency domains. Our contribution
extends the Variogram method developed by the geostatistics literature and,
accordingly, it is referred to as the Generalised Variogram method (GVM). In
addition to the theoretical presentation of GVM, we provide experimental
validation in terms of accuracy, consistency with ML and computational
complexity for different kernels using synthetic and real-world data
Approximative Covariance Interpolation
Abstract-When methods of moments are used for identification of power spectral densities, a model is matched to estimated second order statistics such as, e.g., covariance estimates. If the estimates are good there is an infinite family of power spectra consistent with such an estimate and in applications, such as identification, we want to single out the most representative spectrum. We choose a prior spectral density to represent a priori information, and the spectrum closest to it in a given quasi-distance is determined. However, if the estimates are based on few data, or the model class considered is not consistent with the process considered, it may be necessary to use an approximative covariance interpolation. Two different types of regularizations are considered in this paper that can be applied on many covariance interpolation based estimation methods
On the Geometry of Maximum Entropy Problems
We show that a simple geometric result suffices to derive the form of the
optimal solution in a large class of finite and infinite-dimensional maximum
entropy problems concerning probability distributions, spectral densities and
covariance matrices. These include Burg's spectral estimation method and
Dempster's covariance completion, as well as various recent generalizations of
the above. We then apply this orthogonality principle to the new problem of
completing a block-circulant covariance matrix when an a priori estimate is
available.Comment: 22 page
- …