Regularized Covariance Matrix Estimation in Complex Elliptically Symmetric Distributions Using the Expected Likelihood Approach - Part 2: The Under-Sampled Case

In the first part of this series of two papers, we extended the expected likelihood approach originally developed in the Gaussian case, to the broader class of complex elliptically symmetric (CES) distributions and complex angular central Gaussian (ACG) distributions. More precisely, we demonstrated that the probability density function (p.d.f.) of the likelihood ratio (LR) for the (unknown) actual scatter matrix \mSigma_{0} does not depend on the latter: it only depends on the density generator for the CES distribution and is distribution-free in the case of ACG distributed data, i.e., it only depends on the matrix dimension $M$ and the number of independent training samples $T$, assuming that $T \geq M$. Additionally, regularized scatter matrix estimates based on the EL methodology were derived. In this second part, we consider the under-sampled scenario ($T \leq M$) which deserves a specific treatment since conventional maximum likelihood estimates do not exist. Indeed, inference about the scatter matrix can only be made in the $T$-dimensional subspace spanned by the columns of the data matrix. We extend the results derived under the Gaussian assumption to the CES and ACG class of distributions. Invariance properties of the under-sampled likelihood ratio evaluated at \mSigma_{0} are presented. Remarkably enough, in the ACG case, the p.d.f. of this LR can be written in a rather simple form as a product of beta distributed random variables. The regularized schemes derived in the first part, based on the EL principle, are extended to the under-sampled scenario and assessed through numerical simulations

The H-test probability distribution revisited: Improved sensitivity

Aims: To provide a significantly improved probability distribution for the H-test for periodicity in X-ray and $\gamma$-ray arrival times, which is already extensively used by the $\gamma$-ray pulsar community. Also, to obtain an analytical probability distribution for stacked test statistics in the case of a search for pulsed emission from an ensemble of pulsars where the significance per pulsar is relatively low, making individual detections insignificant on their own. This information is timely given the recent rapid discovery of new pulsars with the Fermi-LAT t $\gamma$-ray telescope. Methods: Approximately $10^{14}$ realisations of the H-statistic ($H$) for random (white) noise is calculated from a random number generator for which the repitition cycle is $\gg 10^{14}$. From these numbers the probability distribution $P(>H)$ is calculated. Results: The distribution of $H$ is is found to be exponential with parameter $\lambda=0.4$ so that the cumulative probability distribution $P(>H)=\exp{(-\lambda H)}$. If we stack independent values for $H$, the sum of $K$ such values would follow the Erlang-K distribution with parameter $\lambda$ for which the cumulative probability distribution is also a simple analytical expression. Conclusion: Searches for weak pulsars with unknown pulse profile shapes in the Fermi-LAT, Agile or other X-ray data bases should benefit from the {\it H-test} since it is known to be powerful against a broad range of pulse profiles, which introduces only a single statistical trial if only the {\it H-test} is used. The new probability distribution presented here favours the detection of weaker pulsars in terms of an improved sensitivity relative to the previously known distribution.Comment: 4 pages, two figures, to appear in Astronomy and Astrophysics, Letter

Generalization and Equilibrium in Generative Adversarial Nets (GANs)

We show that training of generative adversarial network (GAN) may not have good generalization properties; e.g., training may appear successful but the trained distribution may be far from target distribution in standard metrics. However, generalization does occur for a weaker metric called neural net distance. It is also shown that an approximate pure equilibrium exists in the discriminator/generator game for a special class of generators with natural training objectives when generator capacity and training set sizes are moderate. This existence of equilibrium inspires MIX+GAN protocol, which can be combined with any existing GAN training, and empirically shown to improve some of them.Comment: This is an updated version of an ICML'17 paper with the same title. The main difference is that in the ICML'17 version the pure equilibrium result was only proved for Wasserstein GAN. In the current version the result applies to most reasonable training objectives. In particular, Theorem 4.3 now applies to both original GAN and Wasserstein GA

Adversarial Generation of Natural Language

Generative Adversarial Networks (GANs) have gathered a lot of attention from the computer vision community, yielding impressive results for image generation. Advances in the adversarial generation of natural language from noise however are not commensurate with the progress made in generating images, and still lag far behind likelihood based methods. In this paper, we take a step towards generating natural language with a GAN objective alone. We introduce a simple baseline that addresses the discrete output space problem without relying on gradient estimators and show that it is able to achieve state-of-the-art results on a Chinese poem generation dataset. We present quantitative results on generating sentences from context-free and probabilistic context-free grammars, and qualitative language modeling results. A conditional version is also described that can generate sequences conditioned on sentence characteristics.Comment: 11 pages, 3 figures, 5 table