Lessons from a case of academic misconduc

Hollands, Gareth J

Marteau, Theresa M

Munafò, Marcus R

English

Explore Bristol Research

                          Munafò, M. R., Hollands, G. J., & Marteau, T. M. (2018). Open scienceprevents mindless science. BMJ, 363, [k4309].https://doi.org/10.1136/bmj.k4309Publisher's PDF, also known as Version of recordLink to published version (if available):10.1136/bmj.k4309Link to publication record in Explore Bristol ResearchPDF-documentThis is the final published version of the article (version of record). It first appeared online via BMJ at DOI:10.1136/bmj.k4309. Please refer to any applicable terms of use of the publisher.University of Bristol - Explore Bristol ResearchGeneral rightsThis document is made available in accordance with publisher policies. Please cite only the publishedversion using the reference above. Full terms of use are available:http://www.bristol.ac.uk/pure/about/ebr-termsOpen science prevents mindless science OPEN ACCESSLessons from a case of academic misconductMarcus R Munafò professor 1, Gareth J Hollands senior research associate 2, Theresa M Marteauprofessor 21University of Bristol, Bristol, UK; 2Behaviour and Health Research Unit, University of Cambridge, Cambridge, UKOn 19 September 2018, three JAMA journals retractedpublications of six studies that had Brian Wansink, John Dysonprofessor of marketing at Cornell University, as an author.1 Thisadds to seven previous retractions of his work, making a totalof 13 articles retracted as of 10 October2; others may follow.Many of the problems in the retracted studies came to lightwhen a small group of researchers carried out detailed analysisof several articles coauthored by Wansink.3 A subsequentinvestigation by Cornell found that Wansink committedacademic misconduct, including “misreporting of research data,problematic statistical techniques, failure to properly documentand preserve research results, and inappropriate authorship.”4This raises concerns of—at the least—endemic sloppinessand—at worst—a disproportionate focus on creating compellingnarratives rather than establishing firm observations.Such academic misconduct damages trust in science andscientists, and this may be particularly marked in the Wansinkcase given his profile outside academia. His work, encapsulatedin the book Mindless Eating, influenced wider scientific, policy,and public discourse about how the food environment can bealtered to change our behaviour. He was executive director ofthe USDA Center for Nutrition Policy and Promotion from 2007to 2009, where he led revision of the 2010 Dietary Guidelinesfor America, and the Smarter Lunchrooms strategies he devisedare currently being used in over 29 000 schools.His studies have been included in countless narrative andsystematic reviews, raising the question of the extent to whichthe retraction of 13 of his studies undermines evidence in thisfield. For example, two Cochrane reviews for which two of us(GJH and TMM) are authors, one on the effect of portion,package, and tableware size5 and the other on nutritionallabelling,6 include studies by Wansink. None of the includedstudies has been retracted, leaving the reviews’ results andconclusions unchanged. But the retractions raise questions aboutthe veracity of other studies Wansink has authored.It is tempting to treat this as an isolated case, but retractions arenot uncommon and are increasing in frequency, althoughwhether this reflects increased detection or prevalence isunknown.7 Unfortunately, the responses of institutions to casessuch as that of Wansink tend to focus on allocating blame,avoiding litigation, and ultimately hoping that the whole thingwill blow over. What is lacking is serious reflection byinstitutions—universities, funders, and journal editors—on thesystemic aspects of their cultures and practices that might havecontributed to the problem in the first place. This is in starkcontrast to the aviation industry, which has an enviable safetyrecord: when a disaster occurs, it is forensically examined notonly to identify its proximal cause but also the wider systemthat allowed it to happen.The NHS is beginning to learn from the aviation industry,8 butuniversities are lagging behind. Some of the institutionalprocesses that may contribute to questionable research practices,which in extreme cases can extend to outright fraud, are clear.For example, institutions may shape the behaviour of scientistsin ways that have unintended consequences bydisproportionately rewarding publication and grant income overrobust observations.What can we do? Open research practices provide the researchcommunity—including researchers, universities, funders, andjournal editors—with a framework for minimising academicmisconduct of the kind perpetrated by Wansink.7 Individualresearchers can protect themselves against their own enthusiasm,and the incentives to discover something, by preregistering theirstudy protocols and analysis plans. Journals and funders couldalso require preregistration of study protocols and mandate datasharing, not just for clinical trials but across research areas andmethods. There may be cases where this is not appropriate, butin general, making research workflows transparent and subjectto potential scrutiny should serve as a quality control measure,including ensuring data are thoroughly checked. Institutionscould encourage open research practices by including these ascriteria in job descriptions, supported by training in theseapproaches.Correspondence to: M R Munafò marcus.munafo@bristol.ac.ukNo commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribeBMJ 2018;363:k4309 doi: 10.1136/bmj.k4309 (Published 15 October 2018) Page 1 of 2EditorialsEDITORIALS on 12 March 2019 by guest. Protected by copyright.http://www.bmj.com/BMJ: first published as 10.1136/bmj.k4309 on 15 October 2018. Downloaded from Ultimately what is required is a cultural change. None ofWansink’s retracted articles was single authored. Should thecoauthors bear some responsibility for not identifying andhighlighting concerns? The recommendations of theInternational Committee of Medical Journal Editors state thatall authors should be “accountable for all aspects of the work.”9In practice, however, this is complicated by the realities of longdistance international collaborations, specific technicalcontributions of individual authors, and, critically, hierarchiesand power structures that can make it difficult for juniorresearchers to speak out if they feel concerned about the workof more senior authors.There is no “one size fits all” solution that will apply acrossdisciplines, but the culture of many parts of the researchecosystem needs to change if biomedical science is to have arecord for reliability closer to that of the aviation industry.Competing interests: We have read and understood BMJ policy on declaration ofinterests and declare that MRM has prepared reports for Jisc, a provider of digitalproducts for higher education.Provenance and peer review: Commissioned; not externally peer reviewed.1 JAMA Network retracts 6 articles that included Dr Brian Wansink as author. Press release,19 Sep 2018. https://media.jamanetwork.com/news-item/jama-network-retracts-6-articles-that-included-dr-brian-wansink-as-author/.2 Retraction Watch. Retraction database. http://retractiondatabase.org/RetractionSearch.aspx#?auth%3dWansink3 van der Zee T, Anaya J, Brown NJ. Statistical heartburn: an attempt to digest four pizzapublications from the Cornell food and brand lab. BMC Nutrition2017;3:5410.1186/s40795-017-0167-x.4 Provost issues statement on Wansink academic misconduct investigation. Cornell Chronicle2018 Sep 20. http://news.cornell.edu/stories/2018/09/provost-issues-statement-wansink-academic-misconduct-investigation5 Hollands GJ, Shemilt I, Marteau TM, etal . Portion, package or tableware size for changingselection and consumption of food, alcohol and tobacco. Cochrane Database Syst Rev2015;9:CD011045.263682716 Crockett RA, King SE, Marteau TM, etal . Nutritional labelling for healthier food ornon-alcoholic drink purchasing and consumption. Cochrane Database Syst Rev2018;2:CD009315.294822647 Munafò MR, Nosek BA, Bishop DV, etal . A manifesto for reproducible science. NatureHum Behav 2017;1:002110.1038/s41562-016-0021.8 Kapur N, Parand A, Soukup T, Reader T, Sevdalis N. Aviation and healthcare: acomparative review with implications for patient safety. JRSM Open2015;7:2054270415616548.267708179 International Committee of Medical Journal Editors. Recommendations for the conduct,reporting, editing, and publication of scholarly work in medical journals. 2017. http://www.icmje.org/recommendations/Published by the BMJ Publishing Group Limited. For permission to use (where not alreadygranted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissionsThis is an Open Access article distributed in accordance with the CreativeCommons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others todistribute, remix, adapt, build upon this work non-commercially, and license their derivativeworks on different terms, provided the original work is properly cited and the use isnon-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribeBMJ 2018;363:k4309 doi: 10.1136/bmj.k4309 (Published 15 October 2018) Page 2 of 2EDITORIALS on 12 March 2019 by guest. Protected by copyright.http://www.bmj.com/BMJ: first published as 10.1136/bmj.k4309 on 15 October 2018. Downloaded from 

Open science prevents mindless science

Apollo (Cambridge)

Open science prevents mindless science OPEN ACCESSLessons from a case of academic misconductMarcus R Munafò professor 1, Gareth J Hollands senior research associate 2, Theresa M Marteauprofessor 21University of Bristol, Bristol, UK; 2Behaviour and Health Research Unit, University of Cambridge, Cambridge, UKOn 19 September 2018, three JAMA journals retractedpublications of six studies that had Brian Wansink, John Dysonprofessor of marketing at Cornell University, as an author.1 Thisadds to seven previous retractions of his work, making a totalof 13 articles retracted as of 10 October2; others may follow.Many of the problems in the retracted studies came to lightwhen a small group of researchers carried out detailed analysisof several articles coauthored by Wansink.3 A subsequentinvestigation by Cornell found that Wansink committedacademic misconduct, including “misreporting of research data,problematic statistical techniques, failure to properly documentand preserve research results, and inappropriate authorship.”4This raises concerns of—at the least—endemic sloppinessand—at worst—a disproportionate focus on creating compellingnarratives rather than establishing firm observations.Such academic misconduct damages trust in science andscientists, and this may be particularly marked in the Wansinkcase given his profile outside academia. His work, encapsulatedin the book Mindless Eating, influenced wider scientific, policy,and public discourse about how the food environment can bealtered to change our behaviour. He was executive director ofthe USDA Center for Nutrition Policy and Promotion from 2007to 2009, where he led revision of the 2010 Dietary Guidelinesfor America, and the Smarter Lunchrooms strategies he devisedare currently being used in over 29 000 schools.His studies have been included in countless narrative andsystematic reviews, raising the question of the extent to whichthe retraction of 13 of his studies undermines evidence in thisfield. For example, two Cochrane reviews for which two of us(GJH and TMM) are authors, one on the effect of portion,package, and tableware size5 and the other on nutritionallabelling,6 include studies by Wansink. None of the includedstudies has been retracted, leaving the reviews’ results andconclusions unchanged. But the retractions raise questions aboutthe veracity of other studies Wansink has authored.It is tempting to treat this as an isolated case, but retractions arenot uncommon and are increasing in frequency, althoughwhether this reflects increased detection or prevalence isunknown.7 Unfortunately, the responses of institutions to casessuch as that of Wansink tend to focus on allocating blame,avoiding litigation, and ultimately hoping that the whole thingwill blow over. What is lacking is serious reflection byinstitutions—universities, funders, and journal editors—on thesystemic aspects of their cultures and practices that might havecontributed to the problem in the first place. This is in starkcontrast to the aviation industry, which has an enviable safetyrecord: when a disaster occurs, it is forensically examined notonly to identify its proximal cause but also the wider systemthat allowed it to happen.The NHS is beginning to learn from the aviation industry,8 butuniversities are lagging behind. Some of the institutionalprocesses that may contribute to questionable research practices,which in extreme cases can extend to outright fraud, are clear.For example, institutions may shape the behaviour of scientistsin ways that have unintended consequences bydisproportionately rewarding publication and grant income overrobust observations.What can we do? Open research practices provide the researchcommunity—including researchers, universities, funders, andjournal editors—with a framework for minimising academicmisconduct of the kind perpetrated by Wansink.7 Individualresearchers can protect themselves against their own enthusiasm,and the incentives to discover something, by preregistering theirstudy protocols and analysis plans. Journals and funders couldalso require preregistration of study protocols and mandate datasharing, not just for clinical trials but across research areas andmethods. There may be cases where this is not appropriate, butin general, making research workflows transparent and subjectto potential scrutiny should serve as a quality control measure,including ensuring data are thoroughly checked. Institutionscould encourage open research practices by including these ascriteria in job descriptions, supported by training in theseapproaches.Correspondence to: M R Munafò marcus.munafo@bristol.ac.ukNo commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribeBMJ 2018;363:k4309 doi: 10.1136/bmj.k4309 (Published 15 October 2018) Page 1 of 2EditorialsEDITORIALS on 31 October 2018 by guest. Protected by copyright.http://www.bmj.com/BMJ: first published as 10.1136/bmj.k4309 on 15 October 2018. Downloaded from Ultimately what is required is a cultural change. None ofWansink’s retracted articles was single authored. Should thecoauthors bear some responsibility for not identifying andhighlighting concerns? The recommendations of theInternational Committee of Medical Journal Editors state thatall authors should be “accountable for all aspects of the work.”9In practice, however, this is complicated by the realities of longdistance international collaborations, specific technicalcontributions of individual authors, and, critically, hierarchiesand power structures that can make it difficult for juniorresearchers to speak out if they feel concerned about the workof more senior authors.There is no “one size fits all” solution that will apply acrossdisciplines, but the culture of many parts of the researchecosystem needs to change if biomedical science is to have arecord for reliability closer to that of the aviation industry.Competing interests: We have read and understood BMJ policy on declaration ofinterests and declare that MRM has prepared reports for Jisc, a provider of digitalproducts for higher education.Provenance and peer review: Commissioned; not externally peer reviewed.1 JAMA Network retracts 6 articles that included Dr Brian Wansink as author. Press release,19 Sep 2018. https://media.jamanetwork.com/news-item/jama-network-retracts-6-articles-that-included-dr-brian-wansink-as-author/.2 Retraction Watch. Retraction database. http://retractiondatabase.org/RetractionSearch.aspx#?auth%3dWansink3 van der Zee T, Anaya J, Brown NJ. Statistical heartburn: an attempt to digest four pizzapublications from the Cornell food and brand lab. BMC Nutrition2017;3:5410.1186/s40795-017-0167-x.4 Provost issues statement on Wansink academic misconduct investigation. Cornell Chronicle2018 Sep 20. http://news.cornell.edu/stories/2018/09/provost-issues-statement-wansink-academic-misconduct-investigation5 Hollands GJ, Shemilt I, Marteau TM, etal . Portion, package or tableware size for changingselection and consumption of food, alcohol and tobacco. Cochrane Database Syst Rev2015;9:CD011045.263682716 Crockett RA, King SE, Marteau TM, etal . Nutritional labelling for healthier food ornon-alcoholic drink purchasing and consumption. Cochrane Database Syst Rev2018;2:CD009315.294822647 Munafò MR, Nosek BA, Bishop DV, etal . A manifesto for reproducible science. NatureHum Behav 2017;1:002110.1038/s41562-016-0021.8 Kapur N, Parand A, Soukup T, Reader T, Sevdalis N. Aviation and healthcare: acomparative review with implications for patient safety. JRSM Open2015;7:2054270415616548.267708179 International Committee of Medical Journal Editors. Recommendations for the conduct,reporting, editing, and publication of scholarly work in medical journals. 2017. http://www.icmje.org/recommendations/Published by the BMJ Publishing Group Limited. For permission to use (where not alreadygranted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissionsThis is an Open Access article distributed in accordance with the CreativeCommons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others todistribute, remix, adapt, build upon this work non-commercially, and license their derivativeworks on different terms, provided the original work is properly cited and the use isnon-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribeBMJ 2018;363:k4309 doi: 10.1136/bmj.k4309 (Published 15 October 2018) Page 2 of 2EDITORIALS on 31 October 2018 by guest. Protected by copyright.http://www.bmj.com/BMJ: first published as 10.1136/bmj.k4309 on 15 October 2018. Downloaded from 

Open science prevents mindless science.

https://research-information.bris.ac.uk/files/187176282/bmj.k4309.full.pdf

Open science prevents mindless science.

Abstract

Similar works

Full text

Available Versions

Explore Bristol Research

Apollo (Cambridge)