Estimation of the Number of Sources in Unbalanced Arrays via Information Theoretic Criteria

Estimating the number of sources impinging on an array of sensors is a well known and well investigated problem. A common approach for solving this problem is to use an information theoretic criterion, such as Minimum Description Length (MDL) or the Akaike Information Criterion (AIC). The MDL estimator is known to be a consistent estimator, robust against deviations from the Gaussian assumption, and non-robust against deviations from the point source and/or temporally or spatially white additive noise assumptions. Over the years several alternative estimation algorithms have been proposed and tested. Usually, these algorithms are shown, using computer simulations, to have improved performance over the MDL estimator, and to be robust against deviations from the assumed spatial model. Nevertheless, these robust algorithms have high computational complexity, requiring several multi-dimensional searches. In this paper, motivated by real life problems, a systematic approach toward the problem of robust estimation of the number of sources using information theoretic criteria is taken. An MDL type estimator that is robust against deviation from assumption of equal noise level across the array is studied. The consistency of this estimator, even when deviations from the equal noise level assumption occur, is proven. A novel low-complexity implementation method avoiding the need for multi-dimensional searches is presented as well, making this estimator a favorable choice for practical applications.Comment: To appear in the IEEE Transactions on Signal Processin

Optimum estimation via gradients of partition functions and information measures: a statistical-mechanical perspective

In continuation to a recent work on the statistical--mechanical analysis of minimum mean square error (MMSE) estimation in Gaussian noise via its relation to the mutual information (the I-MMSE relation), here we propose a simple and more direct relationship between optimum estimation and certain information measures (e.g., the information density and the Fisher information), which can be viewed as partition functions and hence are amenable to analysis using statistical--mechanical techniques. The proposed approach has several advantages, most notably, its applicability to general sources and channels, as opposed to the I-MMSE relation and its variants which hold only for certain classes of channels (e.g., additive white Gaussian noise channels). We then demonstrate the derivation of the conditional mean estimator and the MMSE in a few examples. Two of these examples turn out to be generalizable to a fairly wide class of sources and channels. For this class, the proposed approach is shown to yield an approximate conditional mean estimator and an MMSE formula that has the flavor of a single-letter expression. We also show how our approach can easily be generalized to situations of mismatched estimation.Comment: 21 pages; submitted to the IEEE Transactions on Information Theor

Bayesian Estimation for Continuous-Time Sparse Stochastic Processes

We consider continuous-time sparse stochastic processes from which we have only a finite number of noisy/noiseless samples. Our goal is to estimate the noiseless samples (denoising) and the signal in-between (interpolation problem). By relying on tools from the theory of splines, we derive the joint a priori distribution of the samples and show how this probability density function can be factorized. The factorization enables us to tractably implement the maximum a posteriori and minimum mean-square error (MMSE) criteria as two statistical approaches for estimating the unknowns. We compare the derived statistical methods with well-known techniques for the recovery of sparse signals, such as the $\ell_1$ norm and Log ($\ell_1$-$\ell_0$ relaxation) regularization methods. The simulation results show that, under certain conditions, the performance of the regularization techniques can be very close to that of the MMSE estimator.Comment: To appear in IEEE TS

MVG Mechanism: Differential Privacy under Matrix-Valued Query

Differential privacy mechanism design has traditionally been tailored for a scalar-valued query function. Although many mechanisms such as the Laplace and Gaussian mechanisms can be extended to a matrix-valued query function by adding i.i.d. noise to each element of the matrix, this method is often suboptimal as it forfeits an opportunity to exploit the structural characteristics typically associated with matrix analysis. To address this challenge, we propose a novel differential privacy mechanism called the Matrix-Variate Gaussian (MVG) mechanism, which adds a matrix-valued noise drawn from a matrix-variate Gaussian distribution, and we rigorously prove that the MVG mechanism preserves $(\epsilon,\delta)$-differential privacy. Furthermore, we introduce the concept of directional noise made possible by the design of the MVG mechanism. Directional noise allows the impact of the noise on the utility of the matrix-valued query function to be moderated. Finally, we experimentally demonstrate the performance of our mechanism using three matrix-valued queries on three privacy-sensitive datasets. We find that the MVG mechanism notably outperforms four previous state-of-the-art approaches, and provides comparable utility to the non-private baseline.Comment: Appeared in CCS'1

Towards a Realistic Assessment of Multiple Antenna HCNs: Residual Additive Transceiver Hardware Impairments and Channel Aging

Given the critical dependence of broadcast channels by the accuracy of channel state information at the transmitter (CSIT), we develop a general downlink model with zero-forcing (ZF) precoding, applied in realistic heterogeneous cellular systems with multiple antenna base stations (BSs). Specifically, we take into consideration imperfect CSIT due to pilot contamination, channel aging due to users relative movement, and unavoidable residual additive transceiver hardware impairments (RATHIs). Assuming that the BSs are Poisson distributed, the main contributions focus on the derivations of the upper bound of the coverage probability and the achievable user rate for this general model. We show that both the coverage probability and the user rate are dependent on the imperfect CSIT and RATHIs. More concretely, we quantify the resultant performance loss of the network due to these effects. We depict that the uplink RATHIs have equal impact, but the downlink transmit BS distortion has a greater impact than the receive hardware impairment of the user. Thus, the transmit BS hardware should be of better quality than user's receive hardware. Furthermore, we characterise both the coverage probability and user rate in terms of the time variation of the channel. It is shown that both of them decrease with increasing user mobility, but after a specific value of the normalised Doppler shift, they increase again. Actually, the time variation, following the Jakes autocorrelation function, mirrors this effect on coverage probability and user rate. Finally, we consider space division multiple access (SDMA), single user beamforming (SU-BF), and baseline single-input single-output (SISO) transmission. A comparison among these schemes reveals that the coverage by means of SU-BF outperforms SDMA in terms of coverage.Comment: accepted in IEEE TV

Fast matrix computations for functional additive models

It is common in functional data analysis to look at a set of related functions: a set of learning curves, a set of brain signals, a set of spatial maps, etc. One way to express relatedness is through an additive model, whereby each individual function $g_{i}\left(x\right)$ is assumed to be a variation around some shared mean $f(x)$. Gaussian processes provide an elegant way of constructing such additive models, but suffer from computational difficulties arising from the matrix operations that need to be performed. Recently Heersink & Furrer have shown that functional additive model give rise to covariance matrices that have a specific form they called quasi-Kronecker (QK), whose inverses are relatively tractable. We show that under additional assumptions the two-level additive model leads to a class of matrices we call restricted quasi-Kronecker, which enjoy many interesting properties. In particular, we formulate matrix factorisations whose complexity scales only linearly in the number of functions in latent field, an enormous improvement over the cubic scaling of na\"ive approaches. We describe how to leverage the properties of rQK matrices for inference in Latent Gaussian Models