Likelihood Analysis of Power Spectra and Generalized Moment Problems

We develop an approach to spectral estimation that has been advocated by Ferrante, Masiero and Pavon and, in the context of the scalar-valued covariance extension problem, by Enqvist and Karlsson. The aim is to determine the power spectrum that is consistent with given moments and minimizes the relative entropy between the probability law of the underlying Gaussian stochastic process to that of a prior. The approach is analogous to the framework of earlier work by Byrnes, Georgiou and Lindquist and can also be viewed as a generalization of the classical work by Burg and Jaynes on the maximum entropy method. In the present paper we present a new fast algorithm in the general case (i.e., for general Gaussian priors) and show that for priors with a specific structure the solution can be given in closed form.Comment: 17 pages, 4 figure

Time and spectral domain relative entropy: A new approach to multivariate spectral estimation

The concept of spectral relative entropy rate is introduced for jointly stationary Gaussian processes. Using classical information-theoretic results, we establish a remarkable connection between time and spectral domain relative entropy rates. This naturally leads to a new spectral estimation technique where a multivariate version of the Itakura-Saito distance is employed}. It may be viewed as an extension of the approach, called THREE, introduced by Byrnes, Georgiou and Lindquist in 2000 which, in turn, followed in the footsteps of the Burg-Jaynes Maximum Entropy Method. Spectral estimation is here recast in the form of a constrained spectrum approximation problem where the distance is equal to the processes relative entropy rate. The corresponding solution entails a complexity upper bound which improves on the one so far available in the multichannel framework. Indeed, it is equal to the one featured by THREE in the scalar case. The solution is computed via a globally convergent matricial Newton-type algorithm. Simulations suggest the effectiveness of the new technique in tackling multivariate spectral estimation tasks, especially in the case of short data records.Comment: 32 pages, submitted for publicatio

On the well-posedness of multivariate spectrum approximation and convergence of high-resolution spectral estimators

In this paper, we establish the well-posedness of the generalized moment problems recently studied by Byrnes-Georgiou-Lindquist and coworkers, and by Ferrante-Pavon-Ramponi. We then apply these continuity results to prove almost sure convergence of a sequence of high-resolution spectral estimators indexed by the sample size

On the existence of a solution to a spectral estimation problem \emph{\`a la} Byrnes-Georgiou-Lindquist

A parametric spectral estimation problem in the style of Byrnes, Georgiou, and Lindquist was posed in \cite{FPZ-10}, but the existence of a solution was only proved in a special case. Based on their results, we show that a solution indeed exists given an arbitrary matrix-valued prior density. The main tool in our proof is the topological degree theory.Comment: 6 pages of two-column draft, accepted for publication in IEEE-TA

A globally convergent matricial algorithm for multivariate spectral estimation

In this paper, we first describe a matricial Newton-type algorithm designed to solve the multivariable spectrum approximation problem. We then prove its global convergence. Finally, we apply this approximation procedure to multivariate spectral estimation, and test its effectiveness through simulation. Simulation shows that, in the case of short observation records, this method may provide a valid alternative to standard multivariable identification techniques such as MATLAB's PEM and MATLAB's N4SID

Relative entropy and the multi-variable multi-dimensional moment problem

Entropy-like functionals on operator algebras have been studied since the pioneering work of von Neumann, Umegaki, Lindblad, and Lieb. The most well-known are the von Neumann entropy $trace (\rho\log \rho)$ and a generalization of the Kullback-Leibler distance $trace (\rho \log \rho - \rho \log \sigma)$, refered to as quantum relative entropy and used to quantify distance between states of a quantum system. The purpose of this paper is to explore these as regularizing functionals in seeking solutions to multi-variable and multi-dimensional moment problems. It will be shown that extrema can be effectively constructed via a suitable homotopy. The homotopy approach leads naturally to a further generalization and a description of all the solutions to such moment problems. This is accomplished by a renormalization of a Riemannian metric induced by entropy functionals. As an application we discuss the inverse problem of describing power spectra which are consistent with second-order statistics, which has been the main motivation behind the present work.Comment: 24 pages, 3 figure