Olsen, Bjorn R

Shanahan, Daniel R

Journal of Negative Results in BioMedicine

Harvard University - DASH 

 Opening peer-review: the democracy of science  (Article begins on next page)The Harvard community has made this article openly available.Please share how this access benefits you. Your story matters.Citation Shanahan, Daniel R., and Bjorn R Olsen. 2014. “Opening peer-review: the democracy of science.” Journal of Negative Resultsin Biomedicine 13 (1): 2. doi:10.1186/1477-5751-13-2.http://dx.doi.org/10.1186/1477-5751-13-2.Published Version doi:10.1186/1477-5751-13-2Accessed February 19, 2015 3:16:13 PM ESTCitable Link http://nrs.harvard.edu/urn-3:HUL.InstRepos:11879645Terms of Use This article was downloaded from Harvard University's DASHrepository, and is made available under the terms and conditionsapplicable to Other Posted Material, as set forth athttp://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#LAAEDITORIAL Open AccessOpening peer-review: the democracy of scienceDaniel R Shanahan1* and Bjorn R Olsen2Scientific journals have been called the ‘minutes of science’[1]. Born out of the exchange of letters on scientific topicsand results, publication is a way of documenting what wasdone and, particularly in the case of open-access journals,sharing the outcome. Journal publications are consideredauthoritative and are generally used to inform the workof others, be it further research or, in the case of biomedi-cine, in treatment decisions for patients. This makes somekind of quality control all the more important.The ‘gold standard’ for this quality control is peerreview. Described as a form of self-regulation by quali-fied members of a profession, it was first introduced byHenry Oldenburg, the founding editor of PhilosophicalTransactions of the Royal Society, in 1665 as a means ofvetting contributions to the Royal Society of London andhas persisted in various forms ever since [2].Peer review is the evaluation of a piece of work by twoor more people of similar competence to the authors;but, assuming the reviewing process excludes all thoseinvolved in the direct research itself, the persons mostqualified to judge the validity of a submitted researchpaper are precisely those who are the scientist’s closestcompetitors. This means that the review process canbecome adversarial, with referees seeming to see it astheir responsibility to insist on time-consuming additionsand revisions [3,4]. Moreover, under traditional, closedpeer-review policies, the identity of the reviewer iswithheld from the author, presenting them with agreater opportunity to act arbitrarily. It was in an effortto combat this bias that some journals introduceddouble-blind peer-review, whereby the author’sn a m ewas also concealed from the referee. However, researchis a small world and maintaining that blinding oftenproved impossible.So how did peer review, with these intrinsic issues andbiases, become the judicial system of the intellectual world?Simply put, peer review is to science what democracy wasto Churchill –‘ the worst form of government, except allthose other forms that have been tried from time to time.’It has served science well, with a widely-held view that,while it may not be perfect, it is nonetheless far betterthan anything else we have been able to devise. Indeed,the fundamental idea of peer-review seems sound; theissues lie more with the execution. Under closed systems,such as that currently enforced by the Journal of NegativeResults in BioMedicine, there is a lack of transparency ofthe peer-review process and a lack of availability ofevaluative information about published articles to thepublic. Therefore, as of February 2014, the Journal ofNegative Results in BioMedicine will adopt an openpeer-review policy.Articles already published, or those manuscripts currentlysubmitted, will not be affected by this change. However,for all manuscripts submitted during or after February2014, authors will see the reviewers’ names and, if thearticle is published, the reading public will also see whoreviewed the article and how the authors responded.This will be available as part of the pre-publication historyof the published article. The peer review process willtherefore be completely open and transparent, with thepeer reviews being part of the record.Research into the effect of open peer review suggestsnumerous benefits, in particular accountability, fairnessand crediting reviewers for their efforts [5-7]. Further-more, in a recent study, Kowalczuk et al. revealed thatreviewer reports operating under an open peer-reviewsystem were of overall higher quality than those under aclosed system, with higher scores on questions relatingto feedback on the methods (11% higher), constructivenessof the comments (5% higher), and the amount of evidenceprovided to substantiate the comments (9% higher) [8].Despite this, we recognise that there are also negatives.Some (junior) reviewers may feel uncomfortable signing acritical report, especially when recommending rejection[9]. This reluctance also means that more potentialreferees may need to be invited to review a manuscriptopenly than under a closed peer-review system (ParkinEC et al. unpublished observations) [9-11].Reviewing an article is no easy task and many of uswill have faced the situation where it feels we have putmore thought into our review of the article than the* Correspondence: daniel.shanahan@biomedcentral.com1BioMed Central 6th Floor, 236 Gray’s Inn Road, London WC1X 8HB, UKFull list of author information is available at the end of the article© 2014 Shanahan and Olsen; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of theCreative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use,distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons PublicDomain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in thisarticle, unless otherwise stated.Shanahan and Olsen Journal of Negative Results in BioMedicine 2014, 13:2http://www.jnrbm.com/content/13/1/224authors did in designing the study and writing the manu-script. The move towards an open peer-review policywill give credit where it is due, but moreover will providevaluable information to those reading the article, sharingthe referees’ critique of the manuscript and presenting allthe necessary information for them to make an objectiveevaluation for themselves.Author details1BioMed Central 6th Floor, 236 Gray’s Inn Road, London WC1X 8HB, UK.2Department of Cell Biology, Harvard Medical School, 240 LongwoodAvenue, Boston, MA 02115, USA.Received: 20 January 2014 Accepted: 20 January 2014References1. Velterop JJM: Keeping the minutes of science. In Electronic library andvisual information research (ELVIRA 2). Edited by Collier M, Arnold K. MiltonKeynes, Aslib: Proceedings of the second ELVIRA conference at De MontfortUniversity; 1995.2. National Research Council: On Being a Scientist: A Guide to ResponsibleConduct in Research: Third Edition. Washington, DC: The National AcademiesPress; 2009.3. Ploegh H: End the wasteful tyranny of reviewer experiments. Nature 2011,472:391.4. Walbot V: Are we training pit bulls to review our manuscripts? J Biol 2009,8:24.5. Godlee F: Making reviewers visible: openness, accountability, and credit.JAMA 2002, 287:2762–2765.6. Groves T: Is open peer review the fairest system? Yes. BMJ 2010, 341:c6424.7. Pöschl U: Multi-stage open peer review: scientific evaluation integratingthe strengths of traditional peer review with the virtues of transparencyand self-regulation. Front Comput Neurosci 2012, 6:33.8. Kowalczuk MK, Dudbridge F, Nanda S, Harriman SL, Moylan EC: Acomparison of the quality of reviewer reports from author-suggestedreviewers and editor-suggested reviewers in journals operating on openor closed peer review models. F1000 Posters 2013, 4:1252.9. Khan K: Is open peer review the fairest system? No. BMJ 2010, 341:c6425.10. van Rooyen S, Godlee F, Evans S, Black N, Smith R: Effect of open peerreview on quality of reviews and on reviewers’ recommendations: arandomised trial. BMJ 1999, 318:23–27.11. Editors TPSM: Peer review in PloS medicine. PLoS Med 2007, 4:e58.doi:10.1186/1477-5751-13-2Cite this article as: Shanahan and Olsen: Opening peer-review: thedemocracy of science. Journal of Negative Results in BioMedicine2014 13:2.Submit your next manuscript to BioMed Centraland take full advantage of: • Convenient online submission• Thorough peer review• No space constraints or color ﬁgure charges• Immediate publication on acceptance• Inclusion in PubMed, CAS, Scopus and Google Scholar• Research which is freely available for redistributionSubmit your manuscript at www.biomedcentral.com/submitShanahan and Olsen Journal of Negative Results in BioMedicine 2014, 13:2 Page 2 of 2http://www.jnrbm.com/content/13/1/2Published: January 2014

Opening peer-review: the democracy of science

Daniel R Shanahan

Bjorn R Olsen

Springer - Publisher Connector

Shanahan and Olsen Journal of Negative Results in BioMedicine 2014, 13:2
http://www.jnrbm.com/content/13/1/2EDITORIAL Open AccessOpening peer-review: the democracy of science
Daniel R Shanahan1* and Bjorn R Olsen2Scientific journals have been called the ‘minutes of science’
[1]. Born out of the exchange of letters on scientific topics
and results, publication is a way of documenting what was
done and, particularly in the case of open-access journals,
sharing the outcome. Journal publications are considered
authoritative and are generally used to inform the work
of others, be it further research or, in the case of biomedi-
cine, in treatment decisions for patients. This makes some
kind of quality control all the more important.
The ‘gold standard’ for this quality control is peer
review. Described as a form of self-regulation by quali-
fied members of a profession, it was first introduced by
Henry Oldenburg, the founding editor of Philosophical
Transactions of the Royal Society, in 1665 as a means of
vetting contributions to the Royal Society of London and
has persisted in various forms ever since [2].
Peer review is the evaluation of a piece of work by two
or more people of similar competence to the authors;
but, assuming the reviewing process excludes all those
involved in the direct research itself, the persons most
qualified to judge the validity of a submitted research
paper are precisely those who are the scientist’s closest
competitors. This means that the review process can
become adversarial, with referees seeming to see it as
their responsibility to insist on time-consuming additions
and revisions [3,4]. Moreover, under traditional, closed
peer-review policies, the identity of the reviewer is
withheld from the author, presenting them with a
greater opportunity to act arbitrarily. It was in an effort
to combat this bias that some journals introduced
double-blind peer-review, whereby the author’s name
was also concealed from the referee. However, research
is a small world and maintaining that blinding often
proved impossible.
So how did peer review, with these intrinsic issues and
biases, become the judicial system of the intellectual world?
Simply put, peer review is to science what democracy was
to Churchill – ‘the worst form of government, except all
those other forms that have been tried from time to time.’* Correspondence: daniel.shanahan@biomedcentral.com
1BioMed Central 6th Floor, 236 Gray’s Inn Road, London WC1X 8HB, UK
Full list of author information is available at the end of the article
© 2014 Shanahan and Olsen; licensee BioMed
Creative Commons Attribution License (http:/
distribution, and reproduction in any medium
Domain Dedication waiver (http://creativecom
article, unless otherwise stated.It has served science well, with a widely-held view that,
while it may not be perfect, it is nonetheless far better
than anything else we have been able to devise. Indeed,
the fundamental idea of peer-review seems sound; the
issues lie more with the execution. Under closed systems,
such as that currently enforced by the Journal of Negative
Results in BioMedicine, there is a lack of transparency of
the peer-review process and a lack of availability of
evaluative information about published articles to the
public. Therefore, as of February 2014, the Journal of
Negative Results in BioMedicine will adopt an open
peer-review policy.
Articles already published, or those manuscripts currently
submitted, will not be affected by this change. However,
for all manuscripts submitted during or after February
2014, authors will see the reviewers’ names and, if the
article is published, the reading public will also see who
reviewed the article and how the authors responded.
This will be available as part of the pre-publication history
of the published article. The peer review process will
therefore be completely open and transparent, with the
peer reviews being part of the record.
Research into the effect of open peer review suggests
numerous benefits, in particular accountability, fairness
and crediting reviewers for their efforts [5-7]. Further-
more, in a recent study, Kowalczuk et al. revealed that
reviewer reports operating under an open peer-review
system were of overall higher quality than those under a
closed system, with higher scores on questions relating
to feedback on the methods (11% higher), constructiveness
of the comments (5% higher), and the amount of evidence
provided to substantiate the comments (9% higher) [8].
Despite this, we recognise that there are also negatives.
Some (junior) reviewers may feel uncomfortable signing a
critical report, especially when recommending rejection
[9]. This reluctance also means that more potential
referees may need to be invited to review a manuscript
openly than under a closed peer-review system (Parkin
EC et al. unpublished observations) [9-11].
Reviewing an article is no easy task and many of us
will have faced the situation where it feels we have put
more thought into our review of the article than theCentral Ltd. This is an Open Access article distributed under the terms of the
/creativecommons.org/licenses/by/2.0), which permits unrestricted use,
, provided the original work is properly cited. The Creative Commons Public
mons.org/publicdomain/zero/1.0/) applies to the data made available in this
Shanahan and Olsen Journal of Negative Results in BioMedicine 2014, 13:2 Page 2 of 2
http://www.jnrbm.com/content/13/1/224
authors did in designing the study and writing the manu-
script. The move towards an open peer-review policy
will give credit where it is due, but moreover will provide
valuable information to those reading the article, sharing
the referees’ critique of the manuscript and presenting all
the necessary information for them to make an objective
evaluation for themselves.
Author details
1BioMed Central 6th Floor, 236 Gray’s Inn Road, London WC1X 8HB, UK.
2Department of Cell Biology, Harvard Medical School, 240 Longwood
Avenue, Boston, MA 02115, USA.
Received: 20 January 2014 Accepted: 20 January 2014
References
1. Velterop JJM: Keeping the minutes of science. In Electronic library and
visual information research (ELVIRA 2). Edited by Collier M, Arnold K. Milton
Keynes, Aslib: Proceedings of the second ELVIRA conference at De Montfort
University; 1995.
2. National Research Council: On Being a Scientist: A Guide to Responsible
Conduct in Research: Third Edition. Washington, DC: The National Academies
Press; 2009.
3. Ploegh H: End the wasteful tyranny of reviewer experiments. Nature 2011,
472:391.
4. Walbot V: Are we training pit bulls to review our manuscripts? J Biol 2009,
8:24.
5. Godlee F: Making reviewers visible: openness, accountability, and credit.
JAMA 2002, 287:2762–2765.
6. Groves T: Is open peer review the fairest system? Yes. BMJ 2010, 341:c6424.
7. Pöschl U: Multi-stage open peer review: scientific evaluation integrating
the strengths of traditional peer review with the virtues of transparency
and self-regulation. Front Comput Neurosci 2012, 6:33.
8. Kowalczuk MK, Dudbridge F, Nanda S, Harriman SL, Moylan EC: A
comparison of the quality of reviewer reports from author-suggested
reviewers and editor-suggested reviewers in journals operating on open
or closed peer review models. F1000 Posters 2013, 4:1252.
9. Khan K: Is open peer review the fairest system? No. BMJ 2010, 341:c6425.
10. van Rooyen S, Godlee F, Evans S, Black N, Smith R: Effect of open peer
review on quality of reviews and on reviewers’ recommendations: a
randomised trial. BMJ 1999, 318:23–27.
11. Editors TPSM: Peer review in PloS medicine. PLoS Med 2007, 4:e58.
doi:10.1186/1477-5751-13-2
Cite this article as: Shanahan and Olsen: Opening peer-review: the
democracy of science. Journal of Negative Results in BioMedicine
2014 13:2.
Published: January 2014Submit your next manuscript to BioMed Central
and take full advantage of: 
• Convenient online submission
• Thorough peer review
• No space constraints or color ﬁgure charges
• Immediate publication on acceptance
• Inclusion in PubMed, CAS, Scopus and Google Scholar
• Research which is freely available for redistribution
Submit your manuscript at 
www.biomedcentral.com/submit


file:///data/core-remote/dit/data/Springer-OA/pdf/b24/aHR0cDovL2xpbmsuc3ByaW5nZXIuY29tLzEwLjExODYvMTQ3Ny01NzUxLTEzLTIucGRm.pdf

Opening peer-review: the democracy of science

Abstract

Similar works

Full text

Available Versions

Harvard University - DASH

Springer - Publisher Connector