Bayesian Model Comparison With the g-Prior

Model comparison and selection is an important problem in many model-based signal processing applications. Often, very simple information criteria such as the Akaike information criterion or the Bayesian information criterion are used despite their shortcomings. Compared to these methods, Djuric’s asymptotic MAP rule was an improvement, and in this paper we extend the work by Djuric in several ways. Specifically, we consider the elicitation of proper prior distributions, treat the case of real- and complex-valued data simultaneously in a Bayesian framework similar to that considered by Djuric, and develop new model selection rules for a regression model containing both linear and non-linear parameters. Moreover, we use this framework to give a new interpretation of the popular information criteria and relate their performance to the signal-to-noise ratio of the data. By use of simulations, we also demonstrate that our proposed model comparison and selection rules outperform the traditional information criteria both in terms of detecting the true model and in terms of predicting unobserved data. The simulation code is available online

Comparing algorithms and criteria for designing Bayesian conjoint choice experiments.

The recent algorithm to find efficient conjoint choice designs, the RSC-algorithm developed by Sándor and Wedel (2001), uses Bayesian design methods that integrate the D-optimality criterion over a prior distribution of likely parameter values. Characteristic for this algorithm is that the designs satisfy the minimal level overlap property provided the starting design complies with it. Another, more embedded, algorithm in the literature, developed by Zwerina et al. (1996), involves an adaptation of the modified Fedorov exchange algorithm to the multinomial logit choice model. However, it does not take into account the uncertainty about the assumed parameter values. In this paper, we adjust the modified Fedorov choice algorithm in a Bayesian fashion and compare its designs to those produced by the RSC-algorithm. Additionally, we introduce a measure to investigate the utility balances of the designs. Besides the widely used D-optimality criterion, we also implement the A-, G- and V-optimality criteria and look for the criterion that is most suitable for prediction purposes and that offers the best quality in terms of computational effectiveness. The comparison study reveals that the Bayesian modified Fedorov choice algorithm provides more efficient designs than the RSC-algorithm and that the Dand V-optimality criteria are the best criteria for prediction, but the computation time with the V-optimality criterion is longer.A-Optimality; Algorithms; Bayesian design; Bayesian modified Fedorov choice algorithm; Choice; Conjoint choice experiments; Criteria; D-Optimality; Design; Discrete choice experiments; Distribution; Effectiveness; Fashion; G-optimality; Logit; Methods; Model; Multinomial logit; Predictive validity; Quality; Research; RSC-algorithm; Studies; Time; Uncertainty; V-optimality; Value;

Compatibility of Prior Specifications Across Linear Models

Bayesian model comparison requires the specification of a prior distribution on the parameter space of each candidate model. In this connection two concerns arise: on the one hand the elicitation task rapidly becomes prohibitive as the number of models increases; on the other hand numerous prior specifications can only exacerbate the well-known sensitivity to prior assignments, thus producing less dependable conclusions. Within the subjective framework, both difficulties can be counteracted by linking priors across models in order to achieve simplification and compatibility; we discuss links with related objective approaches. Given an encompassing, or full, model together with a prior on its parameter space, we review and summarize a few procedures for deriving priors under a submodel, namely marginalization, conditioning, and Kullback--Leibler projection. These techniques are illustrated and discussed with reference to variable selection in linear models adopting a conventional $g$-prior; comparisons with existing standard approaches are provided. Finally, the relative merits of each procedure are evaluated through simulated and real data sets.Comment: Published in at http://dx.doi.org/10.1214/08-STS258 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Comparison of the Bayesian and maximum likelilhood estimation for Weibull distribution

Problem statement: The Weibull distribution has been widely used especially in the modeling of lifetime event data. It provides a statistical model which has a wide variety of applications in many areas, and the main advantage is its ability in the context of lifetime event, to provide reasonably accurate failure analysis and failure for ecasts especially with extremely small samples. The conventional maximum likelihood method is the usual way to estimate the parameters of a distribution. Bayesian approach has received much attention and in contention with other estimation methods. In this study we explore and compare the performance of the maximum likelihood estimate with the Bayesian estimate for the Weibull distribution. Approach: The maximum likelihood estimation, Bayesian using Jeffrey prior and the extension of Jeffrey prior information for estimating the parameters of Weibull distribution of life time are presented. We explore the performance of these estimators numerically under varyin g conditions. Through the simulation study comparison are made on the performance of these estimators with respect to the Mean Square Error (MSE) and Mean Percentage Error(MPE). Results: For all the varying sample size, several specific values of the scale parameter of the Weibull distribution and for the values specify for the extension of Jeffrey prior, the estimators of the maximum likelihood method result in smaller MSE and MPE compared to Bayesian in majority of the cases. Nevertheless in all cases for both methods the MSE and MPE decrease as sample size increases. Conclusion: Based on the results of this simulation study the Bayesian approach used in the estimating of Weibull parameters is found to be not superior compared to the conventional maximum likelihood method with respect to MSE and MPE values

Empirical statistical modelling for crop yields predictions: bayesian and uncertainty approaches

Includes bibliographical referencesThis thesis explores uncertainty statistics to model agricultural crop yields, in a situation where there are neither sampling observations nor historical record. The Bayesian approach to a linear regression model is useful for predict ion of crop yield when there are quantity data issue s and the model structure uncertainty and the regression model involves a large number of explanatory variables. Data quantity issues might occur when a farmer is cultivating a new crop variety, moving to a new farming location or when introducing a new farming technology, where the situation may warrant a change in the current farming practice. The first part of this thesis involved the collection of data from experts' domain and the elicitation of the probability distributions. Uncertainty statistics, the foundation of uncertainty theory and the data gathering procedures were discussed in detail. We proposed an estimation procedure for the estimation of uncertainty distributions. The procedure was then implemented on agricultural data to fit some uncertainty distributions to five cereal crop yields. A Delphi method was introduced and used to fit uncertainty distributions for multiple experts' data of sesame seed yield. The thesis defined an uncertainty distance and derived a distance for a difference between two uncertainty distributions. We lastly estimated the distance between a hypothesized distribution and an uncertainty normal distribution. Although, the applicability of uncertainty statistics is limited to one sample model, the approach provides a fast approach to establish a standard for process parameters. Where no sampling observation exists or it is very expensive to acquire, the approach provides an opportunity to engage experts and come up with a model for guiding decision making. In the second part, we fitted a full dataset obtained from an agricultural survey of small-scale farmers to a linear regression model using direct Markov Chain Monte Carlo (MCMC), Bayesian estimation (with uniform prior) and maximum likelihood estimation (MLE) method. The results obtained from the three procedures yielded similar mean estimates, but the credible intervals were found to be narrower in Bayesian estimates than confidence intervals in MLE method. The predictive outcome of the estimated model was then assessed using simulated data for a set of covariates. Furthermore, the dataset was then randomly split into two data sets. The informative prior was later estimated from one-half called the "old data" using Ordinary Least Squares (OLS) method. Three models were then fitted onto the second half called the "new data": General Linear Model (GLM) (M1), Bayesian model with a non-informative prior (M2) and Bayesian model with informative prior (M3). A leave-one-outcross validation (LOOCV) method was used to compare the predictive performance of these models. It was found that the Bayesian models showed better predictive performance than M1. M3 (with a prior) had moderate average Cross Validation (CV) error and Cross Validation (CV) standard error. GLM performed worst with least average CV error and highest (CV) standard error among the models. In Model M3 (expert prior), the predictor variables were found to be significant at 95% credible intervals. In contrast, most variables were not significant under models M1 and M2. Also, The model with informative prior had narrower credible intervals compared to the non-information prior and GLM model. The results indicated that variability and uncertainty in the data was reasonably reduced due to the incorporation of expert prior / information prior. We lastly investigated the residual plots of these models to assess their prediction performance. Bayesian Model Average (BMA) was later introduced to address the issue of model structure uncertainty of a single model. BMA allows the computation of weighted average over possible model combinations of predictors. An approximate AIC weight was then proposed for model selection instead of frequentist alternative hypothesis testing (or models comparison in a set of competing candidate models). The method is flexible and easy to interpret instead of raw AIC or Bayesian information criterion (BIC), which approximates the Bayes factor. Zellner's g-prior was considered appropriate as it has widely been used in linear models. It preserves the correlation structure among predictors in its prior covariance. The method also yields closed-form marginal likelihoods which lead to huge computational savings by avoiding sampling in the parameter space as in BMA. We lastly determined a single optimal model from all possible combination of models and also computed the log-likelihood of each model

Constraining ocean diffusivity from the 8.2 ka event

Greenland ice-core data containing the 8.2 ka event are utilized by a model-data intercomparison within the Earth system model of intermediate complexity, CLIMBER-2.3 to investigate their potential for constraining the range of uncertain ocean diffusivity properties. Within a stochastic version of the model (Bauer et al. in Paleoceanography 19:PA3014, 2004) it has been possible to mimic the pronounced cooling of the 8.2 ka event with relatively good accuracy considering the timing of the event in comparison to other modelling exercises. When statistically inferring from the 8.2 ka event on diffusivity the technical difficulty arises to establish the related likelihood numerically per realisation of the uncertain model parameters: while mainstream uncertainty analyses can assume a quasi-Gaussian shape of likelihood, with weather fluctuating around a long term mean, the 8.2 ka event as a highly nonlinear effect precludes such an a priori assumption. As a result of this study the Bayesian Analysis leads to a sharp single-mode likelihood for ocean diffusivity parameters within CLIMBER-2.3. Depending on the prior distribution this likelihood leads to a reduction of uncertainty in ocean diffusivity parameters (e. g. for flat prior uncertainty in the vertical ocean diffusivity parameter is reduced by factor 2). These results highlight the potential of paleo data to constrain uncertain system properties and strongly suggest to make further steps with more complex models and richer data sets to harvest this potential. © The Author(s) 2009

Constraining ocean diffusivity from the 8.2 ka event

Greenland ice-core data containing the 8.2 ka event are utilized by a model-data intercomparison within the Earth system model of intermediate complexity, CLIMBER-2.3 to investigate their potential for constraining the range of uncertain ocean diffusivity properties. Within a stochastic version of the model (Bauer et al. in Paleoceanography 19:PA3014, 2004) it has been possible to mimic the pronounced cooling of the 8.2 ka event with relatively good accuracy considering the timing of the event in comparison to other modelling exercises. When statistically inferring from the 8.2 ka event on diffusivity the technical difficulty arises to establish the related likelihood numerically per realisation of the uncertain model parameters: while mainstream uncertainty analyses can assume a quasi-Gaussian shape of likelihood, with weather fluctuating around a long term mean, the 8.2 ka event as a highly nonlinear effect precludes such an a priori assumption. As a result of this study the Bayesian Analysis leads to a sharp single-mode likelihood for ocean diffusivity parameters within CLIMBER-2.3. Depending on the prior distribution this likelihood leads to a reduction of uncertainty in ocean diffusivity parameters (e. g. for flat prior uncertainty in the vertical ocean diffusivity parameter is reduced by factor 2). These results highlight the potential of paleo data to constrain uncertain system properties and strongly suggest to make further steps with more complex models and richer data sets to harvest this potential. © The Author(s) 2009

Bayesian Model Comparison in Genetic Association Analysis: Linear Mixed Modeling and SNP Set Testing

We consider the problems of hypothesis testing and model comparison under a flexible Bayesian linear regression model whose formulation is closely connected with the linear mixed effect model and the parametric models for SNP set analysis in genetic association studies. We derive a class of analytic approximate Bayes factors and illustrate their connections with a variety of frequentist test statistics, including the Wald statistic and the variance component score statistic. Taking advantage of Bayesian model averaging and hierarchical modeling, we demonstrate some distinct advantages and flexibilities in the approaches utilizing the derived Bayes factors in the context of genetic association studies. We demonstrate our proposed methods using real or simulated numerical examples in applications of single SNP association testing, multi-locus fine-mapping and SNP set association testing