Frequentistic approximations to Bayesian prevision of exchangeable random elements

Given a sequence \xi_1, \xi_2,... of X-valued, exchangeable random elements, let q(\xi^(n)) and p_m(\xi^(n)) stand for posterior and predictive distribution, respectively, given \xi^(n) = (\xi_1,..., \xi_n). We provide an upper bound for limsup b_n d_[[X]](q(\xi^(n)), \delta_\empiricn) and limsup b_n d_[X^m](p_m(\xi^(n)), \empiricn^m), where \empiricn is the empirical measure, b_n is a suitable sequence of positive numbers increasing to +\infty, d_[[X]] and d_[X^m] denote distinguished weak probability distances on [[X]] and [X^m], respectively, with the proviso that [S] denotes the space of all probability measures on S. A characteristic feature of our work is that the aforesaid bounds are established under the law of the \xi_n's, unlike the more common literature on Bayesian consistency, where they are studied with respect to product measures (p_0)^\infty, as p_0 varies among the admissible determinations of a random probability measure

Uniform rates of the Glivenko-Cantelli convergence and their use in approximating Bayesian inferences

This paper deals with the problem of quantifying the approximation a probability measure by means of an empirical (in a wide sense) random probability measure, depending on the first n terms of a sequence of random elements. In Section 2, one studies the range of oscillation near zero of the Wasserstein distance ^{(p)}_{\pms} between \pfrak_0 and \hat{\pfrak}_n, assuming that the \xitil_i's are i.i.d. with \pfrak_0 as common law. Theorem 2.3 deals with the case in which \pfrak_0 is fixed as a generic element of the space of all probability measures on (\rd, \mathscr{B}(\rd)) and \hat{\pfrak}_n coincides with the empirical measure. In Theorem 2.4 (Theorem 2.5, respectively) \pfrak_0 is a d-dimensional Gaussian distribution (an element of a distinguished type of statistical exponential family, respectively) and \hat{\pfrak}_n is another $d$-dimensional Gaussian distribution with estimated mean and covariance matrix (another element of the same family with an estimated parameter, respectively). These new results improve on allied recent works (see, e.g., [31]) since they also provide uniform bounds with respect to $n$, meaning that the finiteness of the p-moment of the random variable \sup_{n \geq 1} b_n ^{(p)}_{\pms}(\pfrak_0, \hat{\pfrak}_n) is proved for some suitable diverging sequence b_n of positive numbers. In Section 3, under the hypothesis that the \xitil_i's are exchangeable, one studies the range of the random oscillation near zero of the Wasserstein distance between the conditional distribution--also called posterior--of the directing measure of the sequence, given \xitil_1, \dots, \xitil_n, and the point mass at \hat{\pfrak}_n. In a similar vein, a bound for the approximation of predictive distributions is given. Finally, Theorems from 3.3 to 3.5 reconsider Theorems from 2.3 to 2.5, respectively, according to a Bayesian perspective

ISIPTA'07: Proceedings of the Fifth International Symposium on Imprecise Probability: Theories and Applications

B

Predictive Constructions Based on Measure-Valued Pólya Urn Processes

Measure-valued Pólya urn processes (MVPP) are Markov chains with an additive structure that serve as an extension of the generalized k-color Pólya urn model towards a continuum of pos- sible colors. We prove that, for any MVPP $(\mu_n)_{n ≥ 0}$ on a Polish space $\mathbb{X}$, the normalized sequence $( \mu_n / \mu_n (\mathbb{X}) )_{n \ge 0}$ agrees with the marginal predictive distributions of some random process $(X_n)_{n \ge 1}$. Moreover, $\mu_n = \mu_{n − 1} + R_{X_n}, \ n \ge 1$, where $x \mapsto R_x$ is a random transition kernel on $\mathbb{X}$; thus, if $\mu_{n − 1}$ represents the contents of an urn, then X n denotes the color of the ball drawn with distribution $\mu_{n − 1} / \mu_{n − 1}(\mathbb{X})$ and $R_{X_{n}}$ - the subsequent reinforcement. In the case $R_{X_{n}} = W_n\delta_{X_n}$, for some non-negative random weights $W_1, \ W_2, \$ ... , the process $( X_n )_{n \ge 1}$ is better understood as a randomly reinforced extension of Blackwell and MacQueen’s Pólya sequence. We study the asymptotic properties of the predictive distributions and the empirical frequencies of $( X_n )_{n \ge 1}$ under different assumptions on the weights. We also investigate a generalization of the above models via a randomization of the law of the reinforcement

On Complex Dependence Structures in Bayesian Nonparametrics: a Distance–based Approach

No abstract availableRandom vectors of measures are the main building block to a major portion of Bayesian nonparametric models. The introduction of infinite–dimensional parameter spaces guarantees notable flexibility and generality to the models but makes their treatment and interpretation more demanding. To overcome these issues we seek a deep understanding of infinite–dimensional random objects and their role in modeling complex dependence structures in the data. Comparisons with baseline models play a major role in the learning process and are expressed through the introduction of suitable distances. In particular, we define a distance between the laws of random vectors of measures that builds on the Wasserstein distance and combines intuitive geometric properties with analytical tractability. This is first used to evaluate approximation errors in posterior sampling schemes and then culminates in the definition of a new principled and non model–specific measure of dependence for partial exchangeability, going beyond current measures of linear dependence. The study of dependence is complemented by the investigation of asymptotic properties for partially exchangeable mixture models from a frequentist perspective. We extend Schwartz theory to a multisample framework by relying on natural distances between vectors of densities and leverage it to find optimal contraction rates for a wide class of hierarchical models

On Complex Dependence Structures in Bayesian Nonparametrics: a Distance–based Approach

No abstract availableRandom vectors of measures are the main building block to a major portion of Bayesian nonparametric models. The introduction of infinite–dimensional parameter spaces guarantees notable flexibility and generality to the models but makes their treatment and interpretation more demanding. To overcome these issues we seek a deep understanding of infinite–dimensional random objects and their role in modeling complex dependence structures in the data. Comparisons with baseline models play a major role in the learning process and are expressed through the introduction of suitable distances. In particular, we define a distance between the laws of random vectors of measures that builds on the Wasserstein distance and combines intuitive geometric properties with analytical tractability. This is first used to evaluate approximation errors in posterior sampling schemes and then culminates in the definition of a new principled and non model–specific measure of dependence for partial exchangeability, going beyond current measures of linear dependence. The study of dependence is complemented by the investigation of asymptotic properties for partially exchangeable mixture models from a frequentist perspective. We extend Schwartz theory to a multisample framework by relying on natural distances between vectors of densities and leverage it to find optimal contraction rates for a wide class of hierarchical models

Note on “Frequentistic approximations to Bayesian prevision of exchangeable random elements” [Int. J. Approx. Reason. 78 (2016) 138–152]

This note points some ambiguities in the notation adopted in “Frequentistic approximations to Bayesian prevision of exchangeable random elements” [Int. J. Approx. Reason. 78 (2016) 138–152] and provides the correct way to read those statements and proofs which are affected by the aforesaid ambiguities