Learning about a Categorical Latent Variable under Prior Near-Ignorance

It is well known that complete prior ignorance is not compatible with learning, at least in a coherent theory of (epistemic) uncertainty. What is less widely known, is that there is a state similar to full ignorance, that Walley calls near-ignorance, that permits learning to take place. In this paper we provide new and substantial evidence that also near-ignorance cannot be really regarded as a way out of the problem of starting statistical inference in conditions of very weak beliefs. The key to this result is focusing on a setting characterized by a variable of interest that is latent. We argue that such a setting is by far the most common case in practice, and we show, for the case of categorical latent variables (and general manifest variables) that there is a sufficient condition that, if satisfied, prevents learning to take place under prior near-ignorance. This condition is shown to be easily satisfied in the most common statistical problems.Comment: 15 LaTeX page

Limits of Learning about a Categorical Latent Variable under Prior Near-Ignorance

In this paper, we consider the coherent theory of (epistemic) uncertainty of Walley, in which beliefs are represented through sets of probability distributions, and we focus on the problem of modeling prior ignorance about a categorical random variable. In this setting, it is a known result that a state of prior ignorance is not compatible with learning. To overcome this problem, another state of beliefs, called \emph{near-ignorance}, has been proposed. Near-ignorance resembles ignorance very closely, by satisfying some principles that can arguably be regarded as necessary in a state of ignorance, and allows learning to take place. What this paper does, is to provide new and substantial evidence that also near-ignorance cannot be really regarded as a way out of the problem of starting statistical inference in conditions of very weak beliefs. The key to this result is focusing on a setting characterized by a variable of interest that is \emph{latent}. We argue that such a setting is by far the most common case in practice, and we provide, for the case of categorical latent variables (and general \emph{manifest} variables) a condition that, if satisfied, prevents learning to take place under prior near-ignorance. This condition is shown to be easily satisfied even in the most common statistical problems. We regard these results as a strong form of evidence against the possibility to adopt a condition of prior near-ignorance in real statistical problems.Comment: 27 LaTeX page

A Noninformative Prior on a Space of Distribution Functions

In a given problem, the Bayesian statistical paradigm requires the specification of a prior distribution that quantifies relevant information about the unknowns of main interest external to the data. In cases where little such information is available, the problem under study may possess an invariance under a transformation group that encodes a lack of information, leading to a unique prior---this idea was explored at length by E.T. Jaynes. Previous successful examples have included location-scale invariance under linear transformation, multiplicative invariance of the rate at which events in a counting process are observed, and the derivation of the Haldane prior for a Bernoulli success probability. In this paper we show that this method can be extended, by generalizing Jaynes, in two ways: (1) to yield families of approximately invariant priors, and (2) to the infinite-dimensional setting, yielding families of priors on spaces of distribution functions. Our results can be used to describe conditions under which a particular Dirichlet Process posterior arises from an optimal Bayesian analysis, in the sense that invariances in the prior and likelihood lead to one and only one posterior distribution

Robust Inference of Trees

This paper is concerned with the reliable inference of optimal tree-approximations to the dependency structure of an unknown distribution generating data. The traditional approach to the problem measures the dependency strength between random variables by the index called mutual information. In this paper reliability is achieved by Walley's imprecise Dirichlet model, which generalizes Bayesian learning with Dirichlet priors. Adopting the imprecise Dirichlet model results in posterior interval expectation for mutual information, and in a set of plausible trees consistent with the data. Reliable inference about the actual tree is achieved by focusing on the substructure common to all the plausible trees. We develop an exact algorithm that infers the substructure in time O(m^4), m being the number of random variables. The new algorithm is applied to a set of data sampled from a known distribution. The method is shown to reliably infer edges of the actual tree even when the data are very scarce, unlike the traditional approach. Finally, we provide lower and upper credibility limits for mutual information under the imprecise Dirichlet model. These enable the previous developments to be extended to a full inferential method for trees.Comment: 26 pages, 7 figure

Atmospheric Retrieval for Super-Earths: Uniquely Constraining the Atmospheric Composition with Transmission Spectroscopy

We present a retrieval method based on Bayesian analysis to infer the atmospheric compositions and surface or cloud-top pressures from transmission spectra of exoplanets with general compositions. In this study, we identify what can unambiguously be determined about the atmospheres of exoplanets from their transmission spectra by applying the retrieval method to synthetic observations of the super-Earth GJ 1214b. Our approach to infer constraints on atmospheric parameters is to compute their joint and marginal posterior probability distributions using the MCMC technique in a parallel tempering scheme. A new atmospheric parameterization is introduced that is applicable to general atmospheres in which the main constituent is not known a priori and clouds may be present. Our main finding is that a unique constraint of the mixing ratios of the absorbers and up to two spectrally inactive gases (such as N2 and primordial H2+He) is possible if the observations are sufficient to quantify both (1) the broadband transit depths in at least one absorption feature for each absorber and (2) the slope and strength of the molecular Rayleigh scattering signature. The surface or cloud-top pressure can be quantified if a surface or cloud deck is present. The mean molecular mass can be constrained from the Rayleigh slope or the shapes of absorption features, thus enabling to distinguish between cloudy hydrogen-rich atmospheres and high mean molecular mass atmospheres. We conclude, however, that without the signature of Rayleigh scattering--even with robustly detected infrared absorption features--there is no reliable way to tell if the absorber is the main constituent of the atmosphere or just a minor species with a mixing ratio of <0.1%. The retrieval method leads us to a conceptual picture of which details in transmission spectra are essential for unique characterizations of well-mixed atmospheres.Comment: 23 pages, 13 figures, accepted at ApJ, submitted to ApJ on Nov 4, 201

A Gaussian process framework for modelling instrumental systematics: application to transmission spectroscopy

Transmission spectroscopy, which consists of measuring the wavelength-dependent absorption of starlight by a planet's atmosphere during a transit, is a powerful probe of atmospheric composition. However, the expected signal is typically orders of magnitude smaller than instrumental systematics, and the results are crucially dependent on the treatment of the latter. In this paper, we propose a new method to infer transit parameters in the presence of systematic noise using Gaussian processes, a technique widely used in the machine learning community for Bayesian regression and classification problems. Our method makes use of auxiliary information about the state of the instrument, but does so in a non-parametric manner, without imposing a specific dependence of the systematics on the instrumental parameters, and naturally allows for the correlated nature of the noise. We give an example application of the method to archival NICMOS transmission spectroscopy of the hot Jupiter HD 189733, which goes some way towards reconciling the controversy surrounding this dataset in the literature. Finally, we provide an appendix giving a general introduction to Gaussian processes for regression, in order to encourage their application to a wider range of problems.Comment: 6 figures, 1 table, accepted for publication in MNRA