Chryst, B

van der Linden, S

Sander van der Linden

Breanne Chryst

Directory of Open Access Journals

Frontiers in Applied Mathematics and Statistics

No Need for Bayes Factors: A Fully Bayesian Evidence Synthesis

Frontiers - Publisher Connector

Crossref

Apollo (Cambridge)

OPINIONpublished: 28 June 2017doi: 10.3389/fams.2017.00012Frontiers in Applied Mathematics and Statistics | www.frontiersin.org 1 June 2017 | Volume 3 | Article 12Edited by:Martin Lages,University of Glasgow,United KingdomReviewed by:Francis Tuerlinckx,KU Leuven, Belgium*Correspondence:Sander van der Lindensander.vanderlinden@psychol.cam.ac.ukSpecialty section:This article was submitted toQuantitative Psychology andMeasurement,a section of the journalFrontiers in Applied Mathematics andStatisticsReceived: 17 April 2017Accepted: 12 June 2017Published: 28 June 2017Citation:van der Linden S and Chryst B (2017)No Need for Bayes Factors: A FullyBayesian Evidence Synthesis.Front. Appl. Math. Stat. 3:12.doi: 10.3389/fams.2017.00012No Need for Bayes Factors: A FullyBayesian Evidence SynthesisSander van der Linden 1* and Breanne Chryst 21 Social Decision-Making Lab, Department of Psychology, University of Cambridge, Cambridge, United Kingdom,2Department of Statistics, Yale University, New Haven, CT, United StatesKeywords: reproducibility, Bayesian evidence synthesis, social norms, meta-analysisThe reproducibility of psychological science has become an important topic of debate. In anintriguing recent article, Scheibehenne et al. [1] advocate the use of Bayesian evidence synthesisas a new way to reconcile inconsistent findings in social psychology. As a case in point, theyuse Goldstein et al.’s [2] “hotel towel” study, a highly cited experiment illustrating the power ofsocial norms in encouraging environmental conservation. Yet, because several recent experimentshave failed to reproduce the study’s original findings, Scheibehenne et al. [1] make a strong casefor the need for better evidence synthesis. In doing so, they reveal an important finding, whilenone of the individual studies show strong support for the efficacy of social norms in encouraginghotel towel reuse, when combined and reanalyzed using a Bayesian approach, the studies jointlyprovide strong evidential support for the claim that normative information can encourage positivebehavior change. Accordingly, Scheibehenne et al. [1] conclude that Bayesian evidence synthesis isa promising meta-analytic approach (p. 1045).Although we applaud a Bayesian perspective, it is noteworthy that the approach presented byScheibehenne et al. [1] is not fully Bayesian because it relies almost exclusively on the use of BayesFactors. Indeed, although Bayes Factors are now advocated widely in psychology in place of p-values and null hypothesis significance testing (NHST), Bayes factors suffer from some of the samefundamental flaws [3], and much like the “old” statistics, they do not reveal useful informationabout magnitude and uncertainty [4, 5]. Moreover, although simple rules-of-thumb exist (e.g.,see [6]), what constitutes reliable evidence in favor of the null hypothesis is largely unclear andambiguous. Much like the “p < 0.05” rule of thumb, we are hesitant about the dogmatic use of“automatic” Bayes in psychology—i.e., an uncritical mechanistic interpretation of Bayes factorsindependent of context [7].In addition to their ambiguity, Bayes factors provide an incoherent framework for evidentialsupport because of their high sensitivity to the prior and because they penalize hypothesesthat contain values with low likelihoods [3, 4, 8]. In short, while there are many “Bayesian”approaches, Bayes factors in particular “have no direct foundational meaning to a Bayesian”, asonly posterior probabilities have a proper Bayesian interpretation ([8], p. 56). Although the valueof Bayesian inference has been noted before (e.g., [9, 10]), the approach remains underappreciatedin psychological science. A formal Bayesian estimation approach finds the entire posteriordistribution of a parameter given the data, such as the (pooled) proportions of hotel towel reusein all control and experiment groups. We show that doing so provides a much simpler and moreintuitive Bayesian interpretation of the evidence.Scheibehenne et al. [1] recorded how many participants reused their towel in the control andexperiment groups across all seven experiments. The authors subsequently obtained one-sidedBayes factors for a test of equality of proportions. Using the dataset provided by Scheibehenneet al. [1], we proceed directly to the Bayesian evidence synthesis of the combined total. Assuminga uniform prior distribution (between 0 and 1), we obtain posterior probability distributionsfor both the control and experiment groups using the JAGS procedure in R [11], which usesa Markov Chain Monte Carlo (MCMC) simulation to generate samples from the posteriordistribution. Without the need for Bayes factor hypothesis testing, it is evident that the mean andvariance of the posterior distributions differ substantially and only marginally overlap in the tailsvan der Linden and Chryst Fully Bayesian Evidence SynthesisFIGURE 1 | The posterior probability distributions of the combined control and experiment groups are displayed in (A) based on 10 million samples (hotel towel reusein proportions). (B) Visualizes the posterior distribution of the difference between the control and experiment proportions.(Figure 1A). For example, mean hotel towel reuse is about50% in the control group and 56% in the treatment group(the higher spread in the posterior distribution of the controlgroup reflects greater uncertainty). From the simulation, wevisualized the posterior distribution of the difference betweenthe experiment and control group proportions in a histogram(Figure 1B), from which it is also clear that the density centersaround an average treatment-effect of 6%. The histogram of thedifferences visualizes the frequency of each parameter value in thedistribution. For example, observed differences in the tails, suchas 0 or 15% are extremely unlikely. In fact, we generated a mean-centered credibility interval, which has a clear interpretation:there is a 95% probability that the mean difference between thecombined experiment and control groups falls between 2.5 and9.5%, with an average effect-size of 6%.The social norm data serve as a nice illustrative examplebecause the Bayes Factor and Inference procedures happento converge on a similar result. However, it would be naïveto think that because the two approaches provide a similarconclusion, it matters little what approach researchers use forsynthesizing evidence in a Bayesian fashion (see [4] for asimilar warning). Accordingly, we provide a free online “FullyBayesian Evidence Synthesis” application that allows scholarsto implement the proposed estimation approach and downloadthe density distributions and simulated data. In sum, whileScheibehenne et al. [1] conclude that the data are “37 timesmore likely under the alternative than the null hypothesis” (p.1045), we argue that estimating the full posterior probabilitydistributions of the control and experiment groups (and theirdifferences) is much more intuitive and revealing. For example,it provides more useful information about the parameters (i.e.,magnitude and uncertainty) and speaks better to meta-analyticthinking than point estimates. We acknowledge that alternativepriors, random rather than fixed-effects pooling, and hierarchicalmulti-level models could also be explored in this context. In fact,just as social norms can be leveraged to promote sustainability,we hope to change social norms around Bayesian data analysisin psychological science above and beyond the use of BayesFactors.OPEN PRACTICESAll data and materials have been made publicly available via theOpen Science Framework and can be accessed at: https://osf.io/vceab/. The Fully Bayesian Evidence Synthesis application isavailable online at: https://bre-chryst.shinyapps.io/BayesApp/AUTHOR CONTRIBUTIONSAll authors contributed equally and approved the final version ofthe manuscript.ACKNOWLEDGMENTSWe would like to thank Benjamin Scheibehenne, Sir DavidSpiegelhalter, and Francis Tuerlinckx for their useful feedbackand comments on earlier versions of this manuscript.REFERENCES1. Scheibehenne B, Jamil T, Wagenmakers EJ. Bayesian evidence synthesis canreconcile seemingly inconsistent results: The case of hotel towel reuse. PsycholSci. (2016) 27:1043–6. doi: 10.1177/09567976166440812. Goldstein NJ, Cialdini RB, Griskevicius V. A room with a viewpoint: usingsocial norms to motivate environmental conservation in hotels. J Consum Res.(2008) 35:472–82. doi: 10.1086/5869103. Lavine M, Schervish MJ. Bayes factors: What they are and what they are not.Am Statist. (1999) 53:119–22. doi: 10.1080/00031305.1999.104744434. Kruschke JK, Liddell TM. The Bayesian new statistics: Hypothesis testing,estimation, meta-analysis, and planning from a Bayesian perspective. PsychonBull Rev. (2017). doi: 10.3758/s13423-016-1221-4. [Epub ahead of print].5. van der Linden S, Chryst B. Why the “new statistics” isn’t new. Psychologist(2015) 28:610.6. Jeffreys H. Theory of Probability, 3rd Edn., Oxford, UK: Oxford UniversityPress (1961).7. Gigerenzer G, Marewski JN. Surrogate science: The idol of auniversal method for scientific inference. J Manag. (2015) 41:421–40.doi: 10.1177/0149206314547522Frontiers in Applied Mathematics and Statistics | www.frontiersin.org 2 June 2017 | Volume 3 | Article 12van der Linden and Chryst Fully Bayesian Evidence Synthesis8. Bernardo JM. Integrated objective Bayesian estimation and hypothesistesting. In: J. Bernardo M, Bayarri MJ, Berger JO, editors.Bayesian Statistics 9 Oxford, UK: Oxford University Press (2011).p. 1–68.9. Higgins J, Thompson SG, Spiegelhalter DJ. A re-evaluation ofrandom-effects meta-analysis. J R Stat Soc Ser A (2009) 172:137–59.doi: 10.1111/j.1467-985X.2008.00552.x10. Kruschke JK. Doing Bayesian Data Analysis. San Diego, CA: Academic Press;Elsevier (2011).11. Plummer M. Rjags: Bayesian Graphical Models using MCMC. R packageversion 4-6. (2016). Available online at: http://mcmc-jags.sourceforge.net/Conflict of Interest Statement: The authors declare that the research wasconducted in the absence of any commercial or financial relationships that couldbe construed as a potential conflict of interest.Copyright © 2017 van der Linden and Chryst. This is an open-access articledistributed under the terms of the Creative Commons Attribution License (CCBY). The use, distribution or reproduction in other forums is permitted, providedthe original author(s) or licensor are credited and that the original publicationin this journal is cited, in accordance with accepted academic practice. Nouse, distribution or reproduction is permitted which does not comply with theseterms.Frontiers in Applied Mathematics and Statistics | www.frontiersin.org 3 June 2017 | Volume 3 | Article 12

No Need for Bayes Factors: A Fully Bayesian Evidence Synthesis

Abstract

Similar works

Full text

Available Versions

Directory of Open Access Journals

Frontiers - Publisher Connector

Crossref

Apollo (Cambridge)