COMPARATIVE STUDY OF BAYESIAN AND ORDINARY LEAST SQUARES APPROACHES

Abstract

Frequentist (Classical) and Bayesian are two major approaches to data analysis in statistics, however, the difference is how both see a parameter. Frequentists see a parameter as constant value while the Bayesians see it a random variable. Research work recently has witnessed increase in the application of Bayesian methods to statistical problems and in other fields. For linear regression modelling, frequentists use more often the Ordinary Least Squares (OLS) method despite violation of some assumptions. Bayesian approach can be used when assumptions in linear regression model using OLS are not met. Using two different data sets, an empirical study was performed using both OLS and Bayesian approaches to linear regression modelling. The analysis showed that the resulting linear regression model using OLS does not meet all required assumptions for a good model. The Bayesian approach as an alternative to regression modelling was further established based on results using several criteria such as RMSE, MAPE and MAD. The results showed that linear regression modelling using Bayesian approach is better than Frequentist method using OLS regression modelling