Covariance matrix estimation with heterogeneous samples

We consider the problem of estimating the covariance matrix Mp of an observation vector, using heterogeneous training samples, i.e., samples whose covariance matrices are not exactly Mp. More precisely, we assume that the training samples can be clustered into K groups, each one containing Lk, snapshots sharing the same covariance matrix Mk. Furthermore, a Bayesian approach is proposed in which the matrices Mk. are assumed to be random with some prior distribution. We consider two different assumptions for Mp. In a fully Bayesian framework, Mp is assumed to be random with a given prior distribution. Under this assumption, we derive the minimum mean-square error (MMSE) estimator of Mp which is implemented using a Gibbs-sampling strategy. Moreover, a simpler scheme based on a weighted sample covariance matrix (SCM) is also considered. The weights minimizing the mean square error (MSE) of the estimated covariance matrix are derived. Furthermore, we consider estimators based on colored or diagonal loading of the weighted SCM, and we determine theoretically the optimal level of loading. Finally, in order to relax the a priori assumptions about the covariance matrix Mp, the second part of the paper assumes that this matrix is deterministic and derives its maximum-likelihood estimator. Numerical simulations are presented to illustrate the performance of the different estimation schemes

Knowledge-aided STAP in heterogeneous clutter using a hierarchical bayesian algorithm

This paper addresses the problem of estimating the covariance matrix of a primary vector from heterogeneous samples and some prior knowledge, under the framework of knowledge-aided space-time adaptive processing (KA-STAP). More precisely, a Gaussian scenario is considered where the covariance matrix of the secondary data may differ from the one of interest. Additionally, some knowledge on the primary data is supposed to be available and summarized into a prior matrix. Two KA-estimation schemes are presented in a Bayesian framework whereby the minimum mean square error (MMSE) estimates are derived. The first scheme is an extension of a previous work and takes into account the non-homogeneity via an original relation. {In search of simplicity and to reduce the computational load, a second estimation scheme, less complex, is proposed and omits the fact that the environment may be heterogeneous.} Along the estimation process, not only the covariance matrix is estimated but also some parameters representing the degree of \emph{a priori} and/or the degree of heterogeneity. Performance of the two approaches are then compared using STAP synthetic data. STAP filter shapes are analyzed and also compared with a colored loading technique

Bounds for estimation of covariance matrices from heterogeneous samples

This correspondence derives lower bounds on the mean-square error (MSE) for the estimation of a covariance matrix mbi Mp, using samples mbi Zk,k=1,...,K, whose covariance matrices mbi Mk are randomly distributed around mbi Mp. This framework can be encountered e.g., in a radar system operating in a nonhomogeneous environment, when it is desired to estimate the covariance matrix of a range cell under test, using training samples from adjacent cells, and the noise is nonhomogeneous between the cells. We consider two different assumptions for mbi Mp. First, we assume that mbi Mp is a deterministic and unknown matrix, and we derive the Cramer-Rao bound for its estimation. In a second step, we assume that mbi Mp is a random matrix, with some prior distribution, and we derive the Bayesian bound under this hypothesis

Joint Covariance Estimation with Mutual Linear Structure

We consider the problem of joint estimation of structured covariance matrices. Assuming the structure is unknown, estimation is achieved using heterogeneous training sets. Namely, given groups of measurements coming from centered populations with different covariances, our aim is to determine the mutual structure of these covariance matrices and estimate them. Supposing that the covariances span a low dimensional affine subspace in the space of symmetric matrices, we develop a new efficient algorithm discovering the structure and using it to improve the estimation. Our technique is based on the application of principal component analysis in the matrix space. We also derive an upper performance bound of the proposed algorithm in the Gaussian scenario and compare it with the Cramer-Rao lower bound. Numerical simulations are presented to illustrate the performance benefits of the proposed method

The analysis of very small samples of repeated measurements II: a modified box correction

There is a need for appropriate methods for the analysis of very small samples of continuous repeated measurements. A key feature of such analyses is the role played by the covariance matrix of the repeated observations. When subjects are few it can be difficult to assess the fit of parsimonious structures for this matrix, while the use of an unstructured form may lead to a serious lack of power. The Kenward-Roger adjustment is now widely adopted as a means of providing an appropriate inferences in small samples, but does not perform adequately in very small samples. Adjusted tests based on the empirical sandwich estimator can be constructed that have good nominal properties, but are seriously underpowered. Further, when such data are incomplete, or unbalanced, or non-saturated mean models are used, exact distributional results do not exist that justify analyses with any sample size. In this paper, a modification of Box's correction applied to a linear model based $F$-statistic is developed for such small sample settings and is shown to have both the required nominal properties and acceptable power across a range of settings for repeated measurements

Knowledge-aided covariance matrix estimation and adaptive detection in compound-Gaussian noise

We address the problem of adaptive detection of a signal of interest embedded in colored noise modeled in terms of a compound-Gaussian process. The covariance matrices of the primary and the secondary data share a common structure while having different power levels. A Bayesian approach is proposed here, where both the power levels and the structure are assumed to be random, with some appropriate distributions. Within this framework we propose MMSE and MAP estimators of the covariance structure and their application to adaptive detection using the NMF test statistic and an optimized GLRT herein derived. Some results, also conducted in comparison with existing algorithms, are presented to illustrate the performances of the proposed algorithms. The relevant result is that the solutions presented herein allows to improve the performance over conventional ones, especially in presence of a small number of training data

Knowledge-aided Bayesian covariance matrix estimation in compound-Gaussian clutter

We address the problem of estimating a covariance matrix R using K samples zk whose covariance matrices are kR, where k are random variables. This problem naturally arises in radar applications in the case of compound-Gaussian clutter. In contrast to the conventional approach which consists in considering R as a deterministic quantity, a knowledge-aided (KA) approach is advocated here, where R is assumed to be a random matrix with some prior distribution. The posterior distribution of R is derived. Since it does not lead to a closed-form expression for the minimum mean-square error (MMSE) estimate of R, both R and k are estimated using a Gibbs-sampling strategy. The maximum a posteriori (MAP) estimator ofR is also derived. It is shown that it obeys an implicit equation which can be solved through an iterative procedure, similarly to the case of deterministic ks, except that KA is now introduced in the iterative scheme. The new estimators are shown to improve over conventional estimators, especially in small sample support

Fast and accurate modelling of longitudinal and repeated measures neuroimaging data

Despite the growing importance of longitudinal data in neuroimaging, the standard analysis methods make restrictive or unrealistic assumptions (e.g., assumption of Compound Symmetry—the state of all equal variances and equal correlations—or spatially homogeneous longitudinal correlations). While some new methods have been proposed to more accurately account for such data, these methods are based on iterative algorithms that are slow and failure-prone. In this article, we propose the use of the Sandwich Estimator (SwE) method which first estimates the parameters of interest with a simple Ordinary Least Square model and second estimates variances/covariances with the “so-called” SwE which accounts for the within-subject correlation existing in longitudinal data. Here, we introduce the SwE method in its classic form, and we review and propose several adjustments to improve its behaviour, specifically in small samples. We use intensive Monte Carlo simulations to compare all considered adjustments and isolate the best combination for neuroimaging data. We also compare the SwE method to other popular methods and demonstrate its strengths and weaknesses. Finally, we analyse a highly unbalanced longitudinal dataset from the Alzheimer's Disease Neuroimaging Initiative and demonstrate the flexibility of the SwE method to fit within- and between-subject effects in a single model. Software implementing this SwE method has been made freely available at http://warwick.ac.uk/tenichols/SwE

Knowledge-aided bayesian detection in heterogeneous environments

We address the problem of detecting a signal of interest in the presence of noise with unknown covariance matrix, using a set of training samples. We consider a situation where the environment is not homogeneous, i.e., when the covariance matrices of the primary and the secondary data are different. A knowledge-aided Bayesian framework is proposed, where these covariance matrices are considered as random, and some information about the covariance matrix of the training samples is available. Within this framework, the maximum a priori (MAP) estimate of the primary data covariance matrix is derived. It is shown that it amounts to colored loading of the sample covariance matrix of the secondary data. The MAP estimate is in turn used to yield a Bayesian version of the adaptive matched filter. Numerical simulations illustrate the performance of this detector, and compare it with the conventional adaptive matched filter