Performance Bounds for Parameter Estimation under Misspecified Models: Fundamental findings and applications

Inferring information from a set of acquired data is the main objective of any signal processing (SP) method. In particular, the common problem of estimating the value of a vector of parameters from a set of noisy measurements is at the core of a plethora of scientific and technological advances in the last decades; for example, wireless communications, radar and sonar, biomedicine, image processing, and seismology, just to name a few. Developing an estimation algorithm often begins by assuming a statistical model for the measured data, i.e. a probability density function (pdf) which if correct, fully characterizes the behaviour of the collected data/measurements. Experience with real data, however, often exposes the limitations of any assumed data model since modelling errors at some level are always present. Consequently, the true data model and the model assumed to derive the estimation algorithm could differ. When this happens, the model is said to be mismatched or misspecified. Therefore, understanding the possible performance loss or regret that an estimation algorithm could experience under model misspecification is of crucial importance for any SP practitioner. Further, understanding the limits on the performance of any estimator subject to model misspecification is of practical interest. Motivated by the widespread and practical need to assess the performance of a mismatched estimator, the goal of this paper is to help to bring attention to the main theoretical findings on estimation theory, and in particular on lower bounds under model misspecification, that have been published in the statistical and econometrical literature in the last fifty years. Secondly, some applications are discussed to illustrate the broad range of areas and problems to which this framework extends, and consequently the numerous opportunities available for SP researchers.Comment: To appear in the IEEE Signal Processing Magazin

Fourier Analysis of Gapped Time Series: Improved Estimates of Solar and Stellar Oscillation Parameters

Quantitative helio- and asteroseismology require very precise measurements of the frequencies, amplitudes, and lifetimes of the global modes of stellar oscillation. It is common knowledge that the precision of these measurements depends on the total length (T), quality, and completeness of the observations. Except in a few simple cases, the effect of gaps in the data on measurement precision is poorly understood, in particular in Fourier space where the convolution of the observable with the observation window introduces correlations between different frequencies. Here we describe and implement a rather general method to retrieve maximum likelihood estimates of the oscillation parameters, taking into account the proper statistics of the observations. Our fitting method applies in complex Fourier space and exploits the phase information. We consider both solar-like stochastic oscillations and long-lived harmonic oscillations, plus random noise. Using numerical simulations, we demonstrate the existence of cases for which our improved fitting method is less biased and has a greater precision than when the frequency correlations are ignored. This is especially true of low signal-to-noise solar-like oscillations. For example, we discuss a case where the precision on the mode frequency estimate is increased by a factor of five, for a duty cycle of 15%. In the case of long-lived sinusoidal oscillations, a proper treatment of the frequency correlations does not provide any significant improvement; nevertheless we confirm that the mode frequency can be measured from gapped data at a much better precision than the 1/T Rayleigh resolution.Comment: Accepted for publication in Solar Physics Topical Issue "Helioseismology, Asteroseismology, and MHD Connections

Bounds for estimation of covariance matrices from heterogeneous samples

This correspondence derives lower bounds on the mean-square error (MSE) for the estimation of a covariance matrix mbi Mp, using samples mbi Zk,k=1,...,K, whose covariance matrices mbi Mk are randomly distributed around mbi Mp. This framework can be encountered e.g., in a radar system operating in a nonhomogeneous environment, when it is desired to estimate the covariance matrix of a range cell under test, using training samples from adjacent cells, and the noise is nonhomogeneous between the cells. We consider two different assumptions for mbi Mp. First, we assume that mbi Mp is a deterministic and unknown matrix, and we derive the Cramer-Rao bound for its estimation. In a second step, we assume that mbi Mp is a random matrix, with some prior distribution, and we derive the Bayesian bound under this hypothesis

A review of closed-form Cramér-Rao Bounds for DOA estimation in the presence of Gaussian noise under a unified framework

The Cramér-Rao Bound (CRB) for direction of arrival (DOA) estimation has been extensively studied over the past four decades, with a plethora of CRB expressions reported for various parametric models. In the literature, there are different methods to derive a closed-form CRB expression, but many derivations tend to involve intricate matrix manipulations which appear difficult to understand. Starting from the Slepian-Bangs formula and following the simplest derivation approach, this paper reviews a number of closed-form Gaussian CRB expressions for the DOA parameter under a unified framework, based on which all the specific CRB presentations can be derived concisely. The results cover three scenarios: narrowband complex circular signals, narrowband complex noncircular signals, and wideband signals. Three signal models are considered: the deterministic model, the stochastic Gaussian model, and the stochastic Gaussian model with the a priori knowledge that the sources are spatially uncorrelated. Moreover, three Gaussian noise models distinguished by the structure of the noise covariance matrix are concerned: spatially uncorrelated noise with unknown either identical or distinct variances at different sensors, and arbitrary unknown noise. In each scenario, a unified framework for the DOA-related block of the deterministic/stochastic CRB is developed, which encompasses one class of closed-form deterministic CRB expressions and two classes of stochastic ones under the three noise models. Comparisons among different CRBs across classes and scenarios are presented, yielding a series of equalities and inequalities which reflect the benchmark for the estimation efficiency under various situations. Furthermore, validity of all CRB expressions are examined, with some specific results for linear arrays provided, leading to several upper bounds on the number of resolvable Gaussian sources in the underdetermined case

A Fresh Look at the Bayesian Bounds of the Weiss-Weinstein Family

International audienceMinimal bounds on the mean square error (MSE) are generally used in order to predict the best achievable performance of an estimator for a given observation model. In this paper, we are interested in the Bayesian bound of the Weiss–Weinstein family. Among this family, we have Bayesian Cramér-Rao bound, the Bobrovsky–MayerWolf–Zakaï bound, the Bayesian Bhattacharyya bound, the Bobrovsky–Zakaï bound, the Reuven–Messer bound, and the Weiss–Weinstein bound. We present a unification of all these minimal bounds based on a rewriting of the minimum mean square error estimator (MMSEE) and on a constrained optimization problem. With this approach, we obtain a useful theoretical framework to derive new Bayesian bounds. For that purpose, we propose two bounds. First, we propose a generalization of the Bayesian Bhattacharyya bound extending the works of Bobrovsky, Mayer–Wolf, and Zakaï. Second, we propose a bound based on the Bayesian Bhattacharyya bound and on the Reuven–Messer bound, representing a generalization of these bounds. The proposed bound is the Bayesian extension of the deterministic Abel bound and is found to be tighter than the Bayesian Bhattacharyya bound, the Reuven–Messer bound, the Bobrovsky–Zakaï bound, and the Bayesian Cramér–Rao bound. We propose some closed-form expressions of these bounds for a general Gaussian observation model with parameterized mean. In order to illustrate our results, we present simulation results in the context of a spectral analysis problem

Matched direction detectors and estimators for array processing with subspace steering vector uncertainties

In this paper, we consider the problem of estimating and detecting a signal whose associated spatial signature is known to lie in a given linear subspace but whose coordinates in this subspace are otherwise unknown, in the presence of subspace interference and broad-band noise. This situation arises when, on one hand, there exist uncertainties about the steering vector but, on the other hand, some knowledge about the steering vector errors is available. First, we derive the maximum-likelihood estimator (MLE) for the problem and compute the corresponding Cramer-Rao bound. Next, the maximum-likelihood estimates are used to derive a generalized likelihood ratio test (GLRT). The GLRT is compared and contrasted with the standard matched subspace detectors. The performances of the estimators and detectors are illustrated by means of numerical simulations

On the Fisher information matrix for multivariate elliptically contoured distributions

The Slepian-Bangs formula provides a very convenient way to compute the Fisher information matrix (FIM) for Gaussian distributed data. The aim of this letter is to extend it to a larger family of distributions, namely elliptically contoured (EC) distributions. More precisely, we derive a closed-form expression of the FIM in this case. This new expression involves the usual term of the Gaussian FIM plus some corrective factors that depend only on the expectations of some functions of the so-called modular variate. Hence, for most distributions in the EC family, derivation of the FIM from its Gaussian counterpart involves slight additional derivations. We show that the new formula reduces to the Slepian-Bangs formula in the Gaussian case and we provide an illustrative example with Student distributions on how it can be used

Bayesian and Hybrid Cramér–Rao Bounds for the Carrier Recovery Under Dynamic Phase Uncertain Channels

International audience—In this paper, we study Bayesian and hybrid Cramér–Rao bounds (BCRB and HCRB) for the code-aided (CA), the data-aided (DA), and the non-data-aided (NDA) dynamical phase estimation of QAM modulated signals. We address the bounds derivation for both the offline scenario, for which the whole observation frame is used, and the online which only takes into account the current and the previous observations. For the CA scenario we show that the computation of the Bayesian information matrix (BIM) and of the hybrid information matrix (HIM) is NP hard. We then resort to the belief-propagation (BP) algorithm or to the Bahl–Cocke–Jelinek–Raviv (BCJR) algorithm to obtain some approximate values. Moreover, in order to avoid the calculus of the inverse of the BIM and of the HIM, we present some closed form expressions for the various CRBs, which greatly reduces the computation complexity. Finally, some simulations allow us to compare the possible improvements enabled by the offline and the CA scenarios. Index Terms—Bayesian Cramér–Rao bound (BCRB), code-aided (CA) bound, data-aided (DA) bound, dynam-ical phase estimation, hybrid Cramér–Rao bound (HCRB), non-data-aided (NDA), offline, online