Bosch, F.

Bühler, P.

Faestermann, T.

Kienle, P.

Litvinov, Yury

Sanjari, Mohammad Shahab

Shubina, D.

Winckler, N.

GSI Repository

Further insight into Bayesian and Akaike information criteria of the EC-decayrate oscillations∗N. Winckler1,2, F. Bosch1, P. Bu¨hler3, T. Faestermann4, P. Kienle† 4, Yu.A. Litvinov1,2, M.S. Sanjari1,5,D. Shubina1,2, and the Two-Body Weak Decay collaboration11GSI, Darmstadt; 2MPI-K, Heidelberg; 3SMI, Vienna; 4TU, Munich; 5EMMI, DarmstadtIntroductionWe discuss the Akaike and Bayesian information crite-ria obtained from the model and data set presented in [1].Both criteria are model-selection methods that consists inpenalizing the log-likelihood as a function of the parame-ters: ICi = −2 log(L(θˆi|data,Mi)) + AnKi. The Akaikeinformation criterion (AIC) is a measure of the relativegoodness of fit of a statistical model. Under appropriateconditions, the model that minimizes the AIC correspondsto the one that minimizes the Kullback-Leibler divergencewith respect to the true unknown distribution.The Bayesian Information Criterion (BIC) stems fromBayesian probabilities. If proper conditions are satisfied,it is twice the negative logarithm of the marginal likeli-hood, i.e. −2 log(P (data|Mi)). From the BIC differenceof model Mi and Mj , i.e. ΔBICi, an approximation of theBayes’ factor : Bi,j ≈ exp(−ΔBICi/2) can be obtained.In this sense, the minimum BIC corresponds to the bestmodel describing the data.ResultsThe AIC and BIC values have been obtained from un-binned maximum likelihood method [2] and are thus freefrom any approximation. The AIC and BIC values as wellas their differences and weights are listed in the table. Notethat ΔAIC and ΔBIC values in the table are defined withrespect to the minimum IC value : ΔIC = ICi − ICmin.The Bayesian and Akaike weights are both defined for twomodels Mi,Mj as wi = e−ΔICi/2e−ΔICi/2+e−ΔICj/2. We observethat the AIC and BIC model selection methods can lead todifferent conclusion depending on the data.Discussion: AIC vs BICIt is a priori not trivial to choose between the AIC andBIC as they rely on various assumptions and asymptoticapproximations, which in both cases are considered unre-alistic [3]. Usually AIC prefers complex models and BICsimpler ones [3]. Although information criteria stem fromdifferent paradigms, the decision making of choosing M 1∗Work supported in part by the DFG cluster of excellence “Originand Structure of the Universe” of the Technische Universita¨t Mu¨nchen,the Helmholtz-CAS Joint Research Group HCJRG-108, the Alliance Pro-gram of the Helmholtz Association (HA216/EMMI), and the HIC-for-FAIR through HGS-HIRE† deceased at January 29, 2013EC data β+ data EC data(245 MHz res.) (245 MHz res.) (cap. pick-up)N 3616 2912 2989AIC0 28683.5 22711.2 23718.4AIC1 28674.7 22710.4 23689.4ΔAIC0 8.78 0.8 29w0 1.2 % 40.1 % 5× 10−5%w1 98.8% 59.9% 100(1− w0)%BIC0 28689.67 22717.1 23724.4BIC1 28699.26 22734.3 23713.4ΔBIC0 0 0 11ΔBIC1 9.6 17 0B0,1 121.5 4914.7 0.0041w0 99.2% 99.98% 0.4%w1 0.8% 0.02% 99.6%αAIC ≈ 20% ≈ 20% ≈ 20%αBIC 6.1× 10−3% 8.2× 10−3% 7.96× 10−3%rather than M0 is mathematically equivalent to a likelihoodratio test in rejecting the null when:−2 log(L0/L1) > An(K1 −K0) (1)where the rejection region is given by the right handterm.The likelihood ratio distribution obtained from MC-toys is in very good agreement with a χ24 distribution1.Thus, in the case of AIC, rejecting the null translates inthe frequentist interpretation to a likelihood ratio test at anα level of P (χ24 > 6) = 0.199. In the case of BIC the αlevel depends on the sample size (c.f. αBIC in table). Herewe observe that AIC would have, in the frequentist inter-pretation, a Type I error rate of about 20% while for BICthe error rates decrease as the sample size increases andare all below 10−2%. On the other hand, the Type II errorrates are in general lower for AIC than for BIC [3]. There-fore in situations where Type I error has to be avoided, theBIC is usually prefered. And vice versa, AIC is recom-manded against Type II error. In the case of the resonatorEC-data M1 is favoured if AIC is used and M0 in the caseof the BIC. In order to resolve this issue, a proper alterna-tive would be the computation of the Bayes’ factor fromunbinned likelihood, which would significantly reduce thenumber of assumptions and approximations made.References[1] P. Kienle et al. PLB 726 (2013) 638[2] N. Winckler et al. GSI report 2013[3] J. Dziak et al., Penn. State U.,Tech. Rep. Ser. 12-119 (2012)1From the Wilks theorem one expects a χ23 distribution which is notobtained from MC toys. Assuming the Wilks theorem is valid in our case,αAIC reduces to 11.1% and αBIC reduce slightly as well.GSI SCIENTIFIC REPORT 2013 PNI-AP-06doi:10.15120/GR-2014-1-PNI-AP-06 163

Further insight into Bayesian and Akaike information criteria of the EC-decay rate oscillations

http://repository.gsi.de/record/67869/files/PNI-AP-06.pdf

Further insight into Bayesian and Akaike information criteria of the EC-decay rate oscillations

Abstract

Similar works

Full text

Available Versions

GSI Repository