Multivariate limited translation hierarchical Bayes estimators

Based on the notion of predictive influence functions, the paper develops multivariate limited translation hierarchical Bayes estimators of the normal mean vector which serve as a compromise between the hierarchical Bayes and maximum likelihood estimators. The paper demonstrates the superiority of the limited translation estimators over the usual hierarchical Bayes estimators in terms of the frequentist risks when the true parameter to be estimated departs widely from the grand average of all the parameters

Small Area Shrinkage Estimation

The need for small area estimates is increasingly felt in both the public and private sectors in order to formulate their strategic plans. It is now widely recognized that direct small area survey estimates are highly unreliable owing to large standard errors and coefficients of variation. The reason behind this is that a survey is usually designed to achieve a specified level of accuracy at a higher level of geography than that of small areas. Lack of additional resources makes it almost imperative to use the same data to produce small area estimates. For example, if a survey is designed to estimate per capita income for a state, the same survey data need to be used to produce similar estimates for counties, subcounties and census divisions within that state. Thus, by necessity, small area estimation needs explicit, or at least implicit, use of models to link these areas. Improved small area estimates are found by "borrowing strength" from similar neighboring areas.Comment: Published in at http://dx.doi.org/10.1214/11-STS374 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Shrinkage Estimators in Online Experiments

We develop and analyze empirical Bayes Stein-type estimators for use in the estimation of causal effects in large-scale online experiments. While online experiments are generally thought to be distinguished by their large sample size, we focus on the multiplicity of treatment groups. The typical analysis practice is to use simple differences-in-means (perhaps with covariate adjustment) as if all treatment arms were independent. In this work we develop consistent, small bias, shrinkage estimators for this setting. In addition to achieving lower mean squared error these estimators retain important frequentist properties such as coverage under most reasonable scenarios. Modern sequential methods of experimentation and optimization such as multi-armed bandit optimization (where treatment allocations adapt over time to prior responses) benefit from the use of our shrinkage estimators. Exploration under empirical Bayes focuses more efficiently on near-optimal arms, improving the resulting decisions made under uncertainty. We demonstrate these properties by examining seventeen large-scale experiments conducted on Facebook from April to June 2017

On Application of the Empirical Bayes Shrinkage in Epidemiological Settings

This paper aims to provide direct and indirect evidence on setting up rules for applications of the empirical Bayes shrinkage (EBS), and offers cautionary remarks concerning its applicability. In epidemiology, there is still a lack of relevant criteria in the application of EBS. The bias of the shrinkage estimator is investigated in terms of the sums of errors, squared errors and absolute errors, for both total and individual groups. The study reveals that assessing the underlying exchangeability assumption is important for appropriate use of EBS. The performance of EBS is indicated by a ratio statistic f of the between-group and within-group mean variances. If there are significant differences between the sample means, EBS is likely to produce erratic and even misleading information

Estimateurs bayésiens et intervalles de crédibilité pour des moyennes normales ordonnées en présence d’une contrainte paramétrique incertaine

Abstract: This work concerns parameter estimation for a bivariate normal model where the difference of the two normal means is bounded. A hierarchical prior is considered which allows uncertainty to be introduced in the parametric constraint. We begin by studying the case of a uniform prior on $\theta_1-\theta_2 \geq m$ where $m \sim N(0,\sigma_m^2)$. We show that the Bayes estimator of $\theta_1$ dominates $X_1$ under squared error loss, which extends the known result for $\sigma_m^2=0$; that is, without uncertainty in the constraint. An ad hoc credible set for $\theta_1$ is then presented along with an analysis of its frequentist properties including coverage probability. Finally, the last chapter considers two modifications to the previous model: a parametric restriction of the form $|\theta_1-\theta_2| \leq m$ and the case of unknown variances. [Symboles non conformes]Ce mémoire porte sur l'estimation paramétrique d'un modèle normal bivarié o\`u la différence des deux moyennes normales est bornée. Une loi a priori hiérarchique est considérée, ce qui permet d'introduire de l'incertitude dans la restriction paramétrique. En premier lieu, le cas d'une loi a priori uniforme sur $\theta_1-\theta_2 \geq m$ est étudié o\`u $m \sim N(0,\sigma_m^2)$. On montre que l'estimateur de Bayes de $\theta_1$ domine $X_1$ en termes de perte quadratique lorsque $\theta_1-\theta_2 \geq 0$. Ceci étend le résultat connu pour le cas $\sigma_m^2=0$; c'est-\`a-dire, en absence d'incertitude dans la contrainte. On présente ensuite un intervalle de crédibilité ad hoc pour $\theta_1$ ainsi qu'une analyse de ses propriétés fréquentistes dont la probabilité de recouvrement. Finalement, le dernier chapitre considère deux modifications au modèle précédent: une restriction paramétrique de la forme $|\theta_1-\theta_2| \leq m$ et le cas de variances inconnues. [Symboles non conformes

Towards Machine Wald

The past century has seen a steady increase in the need of estimating and predicting complex systems and making (possibly critical) decisions with limited information. Although computers have made possible the numerical evaluation of sophisticated statistical models, these models are still designed \emph{by humans} because there is currently no known recipe or algorithm for dividing the design of a statistical model into a sequence of arithmetic operations. Indeed enabling computers to \emph{think} as \emph{humans} have the ability to do when faced with uncertainty is challenging in several major ways: (1) Finding optimal statistical models remains to be formulated as a well posed problem when information on the system of interest is incomplete and comes in the form of a complex combination of sample data, partial knowledge of constitutive relations and a limited description of the distribution of input random variables. (2) The space of admissible scenarios along with the space of relevant information, assumptions, and/or beliefs, tend to be infinite dimensional, whereas calculus on a computer is necessarily discrete and finite. With this purpose, this paper explores the foundations of a rigorous framework for the scientific computation of optimal statistical estimators/models and reviews their connections with Decision Theory, Machine Learning, Bayesian Inference, Stochastic Optimization, Robust Optimization, Optimal Uncertainty Quantification and Information Based Complexity.Comment: 37 page

A Hierarchical Bayesian Model for Frame Representation

In many signal processing problems, it may be fruitful to represent the signal under study in a frame. If a probabilistic approach is adopted, it becomes then necessary to estimate the hyper-parameters characterizing the probability distribution of the frame coefficients. This problem is difficult since in general the frame synthesis operator is not bijective. Consequently, the frame coefficients are not directly observable. This paper introduces a hierarchical Bayesian model for frame representation. The posterior distribution of the frame coefficients and model hyper-parameters is derived. Hybrid Markov Chain Monte Carlo algorithms are subsequently proposed to sample from this posterior distribution. The generated samples are then exploited to estimate the hyper-parameters and the frame coefficients of the target signal. Validation experiments show that the proposed algorithms provide an accurate estimation of the frame coefficients and hyper-parameters. Application to practical problems of image denoising show the impact of the resulting Bayesian estimation on the recovered signal quality

Change-point Problem and Regression: An Annotated Bibliography

The problems of identifying changes at unknown times and of estimating the location of changes in stochastic processes are referred to as the change-point problem or, in the Eastern literature, as disorder . The change-point problem, first introduced in the quality control context, has since developed into a fundamental problem in the areas of statistical control theory, stationarity of a stochastic process, estimation of the current position of a time series, testing and estimation of change in the patterns of a regression model, and most recently in the comparison and matching of DNA sequences in microarray data analysis. Numerous methodological approaches have been implemented in examining change-point models. Maximum-likelihood estimation, Bayesian estimation, isotonic regression, piecewise regression, quasi-likelihood and non-parametric regression are among the methods which have been applied to resolving challenges in change-point problems. Grid-searching approaches have also been used to examine the change-point problem. Statistical analysis of change-point problems depends on the method of data collection. If the data collection is ongoing until some random time, then the appropriate statistical procedure is called sequential. If, however, a large finite set of data is collected with the purpose of determining if at least one change-point occurred, then this may be referred to as non-sequential. Not surprisingly, both the former and the latter have a rich literature with much of the earlier work focusing on sequential methods inspired by applications in quality control for industrial processes. In the regression literature, the change-point model is also referred to as two- or multiple-phase regression, switching regression, segmented regression, two-stage least squares (Shaban, 1980), or broken-line regression. The area of the change-point problem has been the subject of intensive research in the past half-century. The subject has evolved considerably and found applications in many different areas. It seems rather impossible to summarize all of the research carried out over the past 50 years on the change-point problem. We have therefore confined ourselves to those articles on change-point problems which pertain to regression. The important branch of sequential procedures in change-point problems has been left out entirely. We refer the readers to the seminal review papers by Lai (1995, 2001). The so called structural change models, which occupy a considerable portion of the research in the area of change-point, particularly among econometricians, have not been fully considered. We refer the reader to Perron (2005) for an updated review in this area. Articles on change-point in time series are considered only if the methodologies presented in the paper pertain to regression analysis