A unified minimax result for restricted parameter spaces

We provide a development that unifies, simplifies and extends considerably a number of minimax results in the restricted parameter space literature. Various applications follow, such as that of estimating location or scale parameters under a lower (or upper) bound restriction, location parameter vectors restricted to a polyhedral cone, scale parameters subject to restricted ratios or products, linear combinations of restricted location parameters, location parameters bounded to an interval with unknown scale, quantiles for location-scale families with parametric restrictions and restricted covariance matrices.Comment: Published in at http://dx.doi.org/10.3150/10-BEJ336 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

J. K. Ghosh's contribution to statistics: A brief outline

Professor Jayanta Kumar Ghosh has contributed massively to various areas of Statistics over the last five decades. Here, we survey some of his most important contributions. In roughly chronological order, we discuss his major results in the areas of sequential analysis, foundations, asymptotics, and Bayesian inference. It is seen that he progressed from thinking about data points, to thinking about data summarization, to the limiting cases of data summarization in as they relate to parameter estimation, and then to more general aspects of modeling including prior and model selection.Comment: Published in at http://dx.doi.org/10.1214/074921708000000011 the IMS Collections (http://www.imstat.org/publications/imscollections.htm) by the Institute of Mathematical Statistics (http://www.imstat.org

Estimation par densités prédictives

L'inférence statistique est un domaine complexe et en constante évolution. Ce mémoire traitera de l'inférence sur la fonction de densité d'une variable aléatoire. Nous partirons de plusieurs résultats connus et développerons une analyse de ces résultats dans le cadre paramétrique avec une approche bayésienne. Nous nous aventurerons aussi dans les problèmes avec espace paramétrique restreint. L'objectif du travail est de trouver les meilleurs estimateurs possibles considérant l'information a priori et l'observation de variables tirées d'une densité faisant intervenir le paramètre. Le chapitre 1 traitera de notions d'inférence bayésienne, de choix de perte évaluant la performance d'un estimateur et possédant des propriétés recherchées. Le chapitre 2 concernera l'estimation ponctuelle du paramètre. En particulier, nous aborderons l'estimateur de James-Stein et trouverons des conditions suffisantes pour la minimaxité et la dominance d'estimateurs en remarquant la forme particulière de ceux-ci. Une condition remontera même à la loi a priori utilisée. Le chapitre 3 établira des liens entre l'estimation ponctuelle et l'estimation par densité prédictive pour le cas multinormal. Des conditions seront aussi établies pour la minimaxité et la dominance. Nous comparerons nos estimateurs à l'estimateur de Bayes découlant d'une loi a priori non informative et démontrerons les résultats par des exemples. Le chapitre 4 considérera le problème dans un cadre plus général où le paramètre d'intérêt pourra être un paramètre de position ou d'échelle. Des liens entre ces deux problèmes seront énoncés et nous trouverons des conditions sur la famille de densités étudiée pour trouver des estimateurs minimax. Quelques exemples concluront cette section. Finalement, le chapitre 5 est l'intégrale de l'article déposé en collaboration avec Tatsuya Kubokawa, Éric Marchand et William E. Strawderman, concernant l'ensemble du problème étudié dans ce mémoire, à savoir l'estimation par densité prédictive dans un espace paramétrique restreint

Sur l'estimation d'un vecteur moyen sous symétrie sphérique et sous contrainte

Ce travail est essentiellement centré sur l'estimation, du point de vue de la théorie de la décision, de la moyenne d'une distribution multidimensionnelle à symétrie sphérique. Sous coût quadratique, nous nous sommes concentrés à développer des classes d'estimateurs au moins aussi bons que les estimateurs usuels, puisque ces derniers tendent à perdre leur performance en dimension élevée et en présence de contraintes sur les paramètres. Dans un premier temps, nous avons considéré les distributions de mélange (par rapport à [sigma][indice supérieur 2]) de lois normales multidimensionnelles N ([théta], [sigma][indice supérieur 2]I[indice inférieur p]), en dimension p supérieure ou égale à 3. Nous avons trouvé une grande classe de lois a priori (généralisées), aussi dans la classe des distributions de mélange de lois normales, qui génèrent des estimateurs de Bayes minimax. Ensuite, pour étendre nos résultats, nous avons considéré les distributions à symétrie sphérique (pas nécessairement mélange de lois normales) avec paramètre d'échelle connu, en dimension supérieure ou égale à 3 et en présence d'un vecteur résiduel. Nous avons obtenu une classe d'estimateurs de Bayes généralisés minimax pour une grande classe de distributions sphériques incluant certaines distributions mélange de lois normales. Dans l'estimation de la moyenne [théta] d'une loi N[indice inférieur p]([théta], I[indice inférieur p]) sous la contrainte [double barre verticale][théta][double barre verticale] [inférieur ou égal] m avec m > 0, une analyse en dimension finie pour comparer les estimateurs linéaires tronqués [delta][indice inférieur a] (0 [plus petit ou égal] a 0, des résultats de dominance de l'estimateur X et de l'estimateur du maximum de vraisemblance [delta][indice inférieur emv] sont développés. En particulier, nous avons montré que le meilleur estimateur équivariant [delta][indice inférieur m] (x , s) = h[indice inférieur m] ([Special characters omitted.]) x pour = [Special characters omitted.] = m domine [delta][indice inférieur emv] lorsque m [plus petit ou égal] [racine carrée]p et que sa troncature [delta][Special characters omitted.] domine [delta][indice inférieur emv] pour tout (m , p)

Stein-Like Estimation and Inference.

The dissertation addresses three issues in the use of Stein-like estimators of the classical normal linear regression model. The St. Louis equation is used to generate out-of-sample forecasts using least squares. These forecasts are compared to those produced by restricted least squares, pretest, and members of a general family of minimax shrinkage estimators using the root-mean-square error criterion. Bootstrap confidence intervals and ellipsoids are constructed which are centered at least squares and James-Stein estimators and their coverage probability and size is explored in a Monte Carlo experiment. A Stein-like estimator of the probit regression model is suggested and its quadratic risk properties are explored in a Monte Carlo experiment

Estimating Parameters under Equality and Inequality Restrictions

The problem of estimating statistical parameters under equality or inequality (order) restrictions has received considerable attention by several researchers due to its vast applications in various physical, industrial and biological experiments. For example, the problem of estimating the common mean of two normal populations when the variances are unknown has a long history and is popularly known as “common mean problem”. This problem is also referred as Meta-Analysis, where samples (data) from multiple sources are combined with a common objective. The “common mean problem” has its origin in the recovery of inter-block information when dealing with Balanced Incomplete Block Designs (BIBDs) problems. In this thesis, we study problem of estimating parameters and quantiles of two or more normal and exponential populations when the parameters are equal or ordered from decision theoretic point of view. In Chapter 1, we give the motivation and do a detailed review of literature for the following problems. In Chapter 2, we discuss some basic definitions and decision theoretic results which are useful in developing the subsequent chapters. In Chapter 3, the problem of estimating the common mean of two normal populations has been considered when it is known a priori that the variances are ordered. Under order restriction on the variances, some new alternative estimators have been proposed including one that uses the maximum likelihood estimator (MLE). These new estimators beat some of the existing popular estimators in terms of stochastic domination as well as Pitman measure of closeness criterion. In Chapter 4, we have considered the problem of estimating quantiles for k( 2) normal populations with a common mean. A general result has been proved which helps in obtaining better estimators. Introducing the principle of invariance, sufficient conditions for improving estimators in certain equivariant classes have been derived. As a consequence some complete class results have been proved. A detailed simulation study has been carried out in order to numerically compare the performances of all the proposed estimators for the cases k = 3 and 4: A similar type of result has also been obtained for estimating the quantile vector. In Chapter 5, we deal with the problem of estimating quantiles and ordered scales of two exponential populations under equality assumption on the location parameters using type-II censored samples. First, we consider the estimation of quantiles of first population when type-II censored samples are available from two exponential populations. Sufficient conditions for improving equivariant estimators have been derived and as a consequence improved estimators have been obtained. A detailed simulation study has been carried out to compare the performances of improved estimators along with some of the existing ones. Further, we deal with the problem of estimating vector of ordered scale parameters. Under order restriction on the scale parameters, we derive the restricted maximum likelihood estimator for the vector parameter. We obtain classes of equivariant estimators and prove some inadmissibility results. Consequently, improved estimators have been derived. Finally a numerical comparison has been done among all the proposed estimators. In Chapter 6, the problem of estimating ordered quantiles of two exponential populations is considered assuming equality of location parameters. Under order restriction, we propose new estimators which are the isotonized version of some baseline estimators. A sufficient condition for improving equivariant estimators are derived under order restriction on quantiles.Consequently, estimators improving upon the baseline estimators are derived. Further, the problem of estimating ordered quantiles of two exponential populations is considered assuming equality of the scale parameters using type-II censored samples. Under order restrictions on the quantiles, isotonized version of some existing estimators have been proposed. Bayes estimators have been derived for the quantiles assuming order restriction on the quantiles. In Chapter 7, we consider the estimation of the common scale parameter of two exponential populations when the location parameters satisfy a simple ordering. Bayes estimators using uniform prior and a conditional inverse gamma prior have been obtained. Finally all the derived estimators have been numerically compared along with some of the existing estimators. In Chapter 8, we give an overall conclusion of the results obtained in the thesis and discuss some of our future research work

Sur une approche décisionnelle pour l'analyse bayésienne et l'estimation de densités prédictives

Cette thèse porte principalement sur l'estimation de densités prédictives pour une inconnue Y à partir d'un observé X, dont les lois de probabilité dépendent d'un même paramètre θ. Elle est le fruit de trois projets sur lesquels j'ai travaillé durant les trois dernières années. Ces projets ont abouti à un article publié, un autre soumis et un Chapitre prometteur. Le premier article, présenté au Chapitre II, a été publié en 2017 dans la revue Journal of Statistical Planning and Inference. Il traite l'étude de densités prédictives pour des modèles Gamma avec restrictions sur le paramètre d'échelle sous la perte Kullback-Leibler. La performance fréquentiste de plusieurs estimateurs est analysée, dont des densités prédictives bayésiennes et des densités prédictives de type plug-in. Des résultats de dominance sont obtenus, ainsi que des densités prédictives minimax. Notamment, on a obtenu une méthode universelle pour améliorer une densité prédictive de type plug-in. Le deuxième article, soumis pour publication récemment, est présenté au Chapitre III. Nous considérons le problème d'estimation d'une densité prédictive pour des modèles de loi normale multivariée, sous une classe de perte de type α-divergence incluant les pertes Kullback-Leibler et Hellinger. Ce travail contient un résultat important sur une stratégie qui donne une classe de densités prédictives dominant une densité prédictive de type plug-in (universellement par rapport à la dimension, l'espace paramétrique, la densité prédictive de type plug-in et la perte choisie). De plus, nos résultats étendent une partie des travaux précédents sur la perte de Kullback-Leibler (Fourdrinier et coll., 2011, Electron. J. Stat), ainsi que ceux portant sur les pertes L1 et L2 intégrées, et sont appliqués aux estimateurs de rétrécissement ou de type Stein de θ. Le troisième projet de cette thèse est le Chapitre 4. Ce dernier traite l'analyse bayésienne pour deux cadres généraux de mélanges avec densités. On donne les structures générales pour les densités a posteriori et les densités prédictives et quelques résultats de dominances. Plusieurs représentations novatrices sont obtenues, notamment pour des mélanges de lois normales, les lois décentrées du chi-deux, Bêta, Fisher, la loi du coefficient de détermination R2 dans un contexte de régression multiple, la loi de Kibble, et les problèmes avec contraintes sur θ, parmi d'autres. Enfin, certaines généralisations et développements sont présentés au chapitre 1.Abstract: This thesis deals mainly with the estimation of predictive densities for an unknown Y from an observed X whose probability laws depend on the same parameter . It is the result of three projects that I have worked on for the last three years. These projects resulted in a published article, another submitted and a promising Chapter. The first article, presented in Chapter II, was published in 2017 in the "Journal of Statistical Planning and Inference". It deals with the study of predictive densities for Gamma models with restrictions on the scale parameter under Kullback-Leibler loss. The frequentist performance of several estimators is analyzed, including Bayesian predictive densities and plug-in predictive densities. Dominance results as well as minimax predictive densities are obtained. In particular, we obtain a universal method to improve upon predictive density of plug-in type. The second article, submitted for publication recently, is presented in Chapter III.We consider the predictive density estimation problem for multivariate normal law models, under a loss class of -divergence including Kullback-Leibler and Hellinger losses. This work contains a significant result on a strategy that gives a class of predictive densities dominating a predictive density of plug-in type (universally compared to the dimension, the parametric space, the predictive density of plug-in type and the chosen loss). In addition, our results extend previous work on the Kullback-Leibler loss (Fourdrinier et al., 2011, J. Electr., J.), as well as those on L1 and L2 losses built-in, and are applied to shrink or Stein estimators of . The third project of the thesis is Chapter 4 which deals with Bayesian analysis for two general frameworks of mixtures with densities. The general structures for the a posteriori densities, the predictive densities and some dominance results are given. Several novel representations are obtained, notably for mixtures of normal laws, the decentered Chi-square laws, Beta, Fisher, the coefficient of determination R2 law in a multiple regression context, the Kibble law, and the problems with constraints on , among others. Finally, some generalizations and developments appear in Chapter 1

Change-point Problem and Regression: An Annotated Bibliography

The problems of identifying changes at unknown times and of estimating the location of changes in stochastic processes are referred to as the change-point problem or, in the Eastern literature, as disorder . The change-point problem, first introduced in the quality control context, has since developed into a fundamental problem in the areas of statistical control theory, stationarity of a stochastic process, estimation of the current position of a time series, testing and estimation of change in the patterns of a regression model, and most recently in the comparison and matching of DNA sequences in microarray data analysis. Numerous methodological approaches have been implemented in examining change-point models. Maximum-likelihood estimation, Bayesian estimation, isotonic regression, piecewise regression, quasi-likelihood and non-parametric regression are among the methods which have been applied to resolving challenges in change-point problems. Grid-searching approaches have also been used to examine the change-point problem. Statistical analysis of change-point problems depends on the method of data collection. If the data collection is ongoing until some random time, then the appropriate statistical procedure is called sequential. If, however, a large finite set of data is collected with the purpose of determining if at least one change-point occurred, then this may be referred to as non-sequential. Not surprisingly, both the former and the latter have a rich literature with much of the earlier work focusing on sequential methods inspired by applications in quality control for industrial processes. In the regression literature, the change-point model is also referred to as two- or multiple-phase regression, switching regression, segmented regression, two-stage least squares (Shaban, 1980), or broken-line regression. The area of the change-point problem has been the subject of intensive research in the past half-century. The subject has evolved considerably and found applications in many different areas. It seems rather impossible to summarize all of the research carried out over the past 50 years on the change-point problem. We have therefore confined ourselves to those articles on change-point problems which pertain to regression. The important branch of sequential procedures in change-point problems has been left out entirely. We refer the readers to the seminal review papers by Lai (1995, 2001). The so called structural change models, which occupy a considerable portion of the research in the area of change-point, particularly among econometricians, have not been fully considered. We refer the reader to Perron (2005) for an updated review in this area. Articles on change-point in time series are considered only if the methodologies presented in the paper pertain to regression analysis