Understanding the role of open peer review and dynamic academic articles

We welcome the commentary by L. Egghe stimulating discussion on our recent article “Natural selection of academic papers” (NSAP) that focuses on an important modern issue at the heart of the scientific enterprise — the open and continuous evaluation and evolution of research. We are also grateful to the editor of Scientometrics for giving us the opportunity to respond to some of the arguments by L. Egghe that we believe are inaccurate or require further comment

Quantifying the volunteer effort of scientific peer reviewing

A survey of 310 reviewers for Monthly Weather Review addresses how much time and effort goes into the peer-review process and provides insight into how reviewers function. Using these data, the individual and collective contributions of volunteer peer reviewers to the peer-review process can be determined. Individually, respondents to the survey review an average of 2 manuscripts a year for Monthly Weather Review, 4 manuscripts a year for AMS journals, and 8 manuscripts a year in total, although some review more than 20 manuscripts a year. Each review takes an average of 9.6 h. Summing the individual contributions of the reviewers, respondents averaged 18 h yr−1 performing reviews for Monthly Weather Review, 36 h yr−1 for AMS journals, and 63 h yr−1 for all journals. The collective time that all of the reviewers put into the peer-review process for all manuscripts submitted to Monthly Weather Review for each year amounts to 362,179 h, or more than 4 years of voluntary labor valued at over $2.34 million. Nearly all respondents (95%) are comfortable with their current load, but only 30% said that they would be willing to perform more reviews. Because the number of submissions to journals has been increasing over time and is unlikely to decrease in the near future, this burden is anticipated to grow. Options for reducing the burden include using fewer reviewers per manuscript, increasing the number of unilateral decisions made by editors, and increasing the size of the reviewer pool (particularly from active retired and early-career scientists).</jats:p

Mapping the landscape: Peer review in computing education research

Peer review is a mainstay of academic publication – indeed, it is the peer-review process that provides much of the publications’ credibility. As the number of computing education conferences and the number of submissions increase, the need for reviewers grows. This report does not attempt to set standards for reviewing; rather, as a first step toward meeting the need for well qualified reviewers, it presents an overview of the ways peer review is used in various venues, both inside computing education and, for com- parison, in closely-related areas outside our field. It considers four key components of peer review in some depth: criteria, the review process, roles and responsibilities, and ethics and etiquette. To do so, it draws on relevant literature, guidance and forms associated with peer review, interviews with journal editors and conference chairs, and a limited survey of the computing education research community. In addition to providing an overview of practice, this report identifies a number of themes running through the discourse that have relevance for decision making about how best to conduct peer review for a given venue

Positioning and power in academic publishing: players, agents and agendas

The field of electronic publishing has grown exponentially in the last two decades, but we are still in the middle of this digital transformation. With technologies coming and going for all kinds of reasons, the distribution of economic, technological and discursive power continues to be negotiated. This book presents the proceedings of the 20th Conference on Electronic Publishing (Elpub), held in Göttingen, Germany, in June 2016. This year’s conference explores issues of positioning and power in academic publishing, and it brings together world leading stakeholders such as academics, practitioners, policymakers, students and entrepreneurs from a wide variety of fields to exchange information and discuss the advent of innovations in the areas of electronic publishing, as well as reflect on the development in the field over the last 20 years. Topics covered in the papers include how to maintain the quality of electronic publications, modeling processes and the increasingly prevalent issue of open access, as well as new systems, database repositories and datasets. This overview of the field will be of interest to all those who work in or make use of electronic publishing