Estimation of multivariate normal covariance and precision matrices in a star-shape model with missing data

AbstractIn this paper, we study the problem of estimating the covariance matrix Σ and the precision matrix Ω (the inverse of the covariance matrix) in a star-shape model with missing data. By considering a type of Cholesky decomposition of the precision matrix Ω=Ψ′Ψ, where Ψ is a lower triangular matrix with positive diagonal elements, we get the MLEs of the covariance matrix and precision matrix and prove that both of them are biased. Based on the MLEs, unbiased estimators of the covariance matrix and precision matrix are obtained. A special group G, which is a subgroup of the group consisting all lower triangular matrices, is introduced. By choosing the left invariant Haar measure on G as a prior, we obtain the closed forms of the best equivariant estimates of Ω under any of the Stein loss, the entropy loss, and the symmetric loss. Consequently, the MLE of the precision matrix (covariance matrix) is inadmissible under any of the above three loss functions. Some simulation results are given for illustration

Alternative Bayesian Estimators for Vector-Autoregressive Models

This paper compares frequentist risks of several Bayesian estimators of the VAR lag parameters and covariance matrix under alternative priors. With the constant prior on the VAR lag parameters, the asymmetric LINEX estimator for the lag parameters does better overall than the posterior mean. The posterior mean of covariance matrix performs well in most cases. The choice of prior has more significant effects on the estimates than the form of estimators. The shrinkage prior on the VAR lag parameters dominates the constant prior, while Yang and Berger's reference prior on the covariance matrix dominates the Jeffreys prior. Estimation of a VAR using the U.S. macroeconomic data reveals significant differences between estimates under the shrinkage and constant priors

Noninformative Priors and Frequentist Risks of Bayesian Estimators of Vector-Autoregressive Models

In this study, we examine posterior properties and frequentist risks of Bayesian estimators based on several non-informative priors in Vector Autoregressive (VAR) models. We prove existence of the posterior distributions and posterior moments under a general class of priors. Using a variety of priors in this class we conduct numerical simulations of posteriors. We find that in most examples Bayesian estimators with a shrinkage prior on the VAR coefficients and the reference prior of Yang and Berger (1994) on the VAR covariance matrix dominate MLE, Bayesian estimators with the diffuse prior, and Bayesian estimators with the prior used in RATS. We also examine the informative Minnesota prior and find that its performance depends on the nature of the data sample and on the tightness of the Minnesota prior. A tightly set Minnesota prior is better when the data generating processes are similar to random walks, but the shrinkage prior or constant prior can be better otherwise

Bayesian Estimator of Vector-Autoregressive Model Under the Entropy Loss

The present study makes two contributions to the Bayesian Vector-Autoregression (VAR) literature. The first contribution is derivation of the Bayesian VAR estimator under the intrinsic entropy loss. The Bayesian estimator, which is distinctly different from the posterior mean, involves the frequentist expectation of a function of VAR variables. We find that the condition that allows for a closed-form expression of the frequentist expectation is violated even when the VAR is stationary, making it difficult to compute the Bayesian estimates via standard Markov Chain Monte Carlo (MCMC) procedures. The second contribution of the paper concerns MCMC simulation of the Bayesian estimator without using the closed-form expression of the frequentist expectation. A novelty of our MCMC algorithms is that they jointly simulate the posteriors of frequentist moments of VAR variables as well as the posteriors of VAR parameters. Numerical simulations show that the algorithms are surprisingly efficient

Bayesian analysis for the Lomax model using noninformative priors

The Lomax distribution is an important member in the distribution family. In this paper, we systematically develop an objective Bayesian analysis of data from a Lomax distribution. Noninformative priors, including probability matching priors, the maximal data information (MDI) prior, Jeffreys prior and reference priors, are derived. The propriety of the posterior under each prior is subsequently validated. It is revealed that the MDI prior and one of the reference priors yield improper posteriors, and the other reference prior is a second-order probability matching prior. A simulation study is conducted to assess the frequentist performance of the proposed Bayesian approach. Finally, this approach along with the bootstrap method is applied to a real data set

Posterior propriety of an objective prior for generalized hierarchical normal linear models

Bayesian Hierarchical models has been widely used in modern statistical application. To deal with the data having complex structures, we propose a generalized hierarchical normal linear (GHNL) model which accommodates arbitrarily many levels, usual design matrices and ‘vanilla’ covariance matrices. Objective hyperpriors can be employed for the GHNL model to express ignorance or match frequentist properties, yet the common objective Bayesian approaches are infeasible or fraught with danger in hierarchical modelling. To tackle this issue, [Berger, J., Sun, D., & Song, C. (2020b). An objective prior for hyperparameters in normal hierarchical models. Journal of Multivariate Analysis, 178, 104606. https://doi.org/10.1016/j.jmva.2020.104606] proposed a particular objective prior and investigated its properties comprehensively. Posterior propriety is important for the choice of priors to guarantee the convergence of MCMC samplers. James Berger conjectured that the resulting posterior is proper for a hierarchical normal model with arbitrarily many levels, a rigorous proof of which was not given, however. In this paper, we complete this story and provide an user-friendly guidance. One main contribution of this paper is to propose a new technique for deriving an elaborate upper bound on the integrated likelihood, but also one unified approach to checking the posterior propriety for linear models. An efficient Gibbs sampling method is also introduced and outperforms other sampling approaches considerably

Fully Bayesian Analysis of the Relevance Vector Machine Classification for Imbalanced Data

Relevance Vector Machine (RVM) is a supervised learning algorithm extended from Support Vector Machine (SVM) based on the Bayesian sparsity model. Compared with the regression problem, RVM classification is difficult to be conducted because there is no closed-form solution for the weight parameter posterior. Original RVM classification algorithm used Newton's method in optimization to obtain the mode of weight parameter posterior then approximated it by a Gaussian distribution in Laplace's method. It would work but just applied the frequency methods in a Bayesian framework. This paper proposes a Generic Bayesian approach for the RVM classification. We conjecture that our algorithm achieves convergent estimates of the quantities of interest compared with the nonconvergent estimates of the original RVM classification algorithm. Furthermore, a Fully Bayesian approach with the hierarchical hyperprior structure for RVM classification is proposed, which improves the classification performance, especially in the imbalanced data problem. By the numeric studies, our proposed algorithms obtain high classification accuracy rates. The Fully Bayesian hierarchical hyperprior method outperforms the Generic one for the imbalanced data classification.Comment: 24 Pages, 3 figures, preprint to submit to Electronic Journal of Statistic

The Effects of Residence Location on Parental Involvement with the School: A Contrast Between Nonmetropolitan Rural and Other Communities

Educational research has long noted the impact of parental involvement with the school on a student\u27s educational success. Despite decades of research, only a few studies have attempted to identify factors that account for variations in parental involvement. In this study, we have employed Coleman\u27s notion of social capital to study the effects of family structure and residence location on parental participation in school related activities. Based on a large stratified sample of Missouri parents, our analyses have demonstrated that parents from dual-parent families and parents who have lived in a school district for a long period of time tend to participate more than their respective counterparts. Further, parents living in nonmetropolitan-rural areas participate in school activities more than those who live in other communities, net of effects of parents\u27 social and demographic characteristics. Also, parents\u27 socioeconomic status (SES) exerts a greater impact on involvement in nonmetropolitan-rural than in other types of communities. Our analysis has concluded that favorable family structures and rural residence location facilitate parental involvement with the school