Is there a need for systematic education on peer-reviewing in Serbia?

Velika brzina stvaranja naučnih informacija u poslednjih nekoliko godina nameće potrebu za povećanjem broja stručnih recenzenata, posebno što recenziranje dobija novu dimenziju važnosti i u svetlu rastućeg broja prepoznatih plagijata i izmisljenih rezultata. Potencijalno rešenje potrebe za povećanjem broja recenzenata je sistematska edukacija doktoranada i mladih istraživača, posebno u zemljama razvoju. Prema podacima sa četiri državna univerziteta u Srbiji (u Beogradu, Novom Sadu, Nišu i Kragujevcu) i jednog privatnog univerziteta (Univerzitet Singidunum), postoji niz doktorskih kurseva koji studente obrazuju o načinu sprovođenja istraživanja, nekoliko o tome kako da pišu naučne publikacije, ali samo jedan od njih skromno osposobljava studente za recenziranje naučnih radova. Dakle, U Srbiji ogromni potencijal doktoranada i mladih istraživača nije pravilno prepoznat i usmeren ka stvaranju kvalifikovane populacije recenzenata. Sa idejom da doprinesemo svetskim trendovima na polju unapređenja recenziranja, predstavili smo pilot seminar na univerzitetima u Beogradu, Novom Sadu, Nišu i Kragujevcu u Srbiji u periodu od oktobra 2017. do aprila 2018. godine. Seminar u organizaciji Centra za promociju nauke obuhvatio je četiri predavanja koja su se bavila: (i) osnovnim aspektima procesa recenziranja, (ii) etičkim pitanjima, (iii) društvenim značajem i (iv) priznavanjem doprinosa recenzenata. Od ukupnog broja polaznika koji je iznosio 275, većina (84 %) se sastojala od doktoranada i mladih istraživača. Nakon seminara, učesnici su zamoljeni da izvrše test recenziju i popune anketu o kvalitetu seminara. Izveštaji o recenziji primljeni su od 82 osobe (od 275 polaznika). Od tri greške koje su namerno unete u tekst, sve greške primetilo je manje od 10 % ispitanika. Analiza ankete (160 odgovora) pokazala je da je 92% ispitanika izjavilo da su stekli nova znanja tokom seminara i ohrabrilo nastavak takve edukacije. Seminar je ponovljen 2019. g. na inicijativu Narodne biblioteke Srbije povodom globalnog događaja "Nedelja recenziranja 2019" i od anketiranih 105 novih polaznika, 94 % je iznelo stav o korisnosti obrazovanja o recenziranju i podržalo ideju o jačanju akademskih kompetencija za recenziranje kroz uvođenje obaveznog predmeta tokom doktorskih studija u Srbiji. Nadamo se da će se predstavnici akademske zajednice iz različitih naučnih disciplina pridružiti ovoj inicijativi i doprineti formiranju kurseva o recenziranju koji odgovaraju specifičnim potrebama svake naučne oblasti

The Miracle of Peer Review and Development in Science: An Agent-Based Model

It is not easy to rationalize how peer review, as the current grassroots of science, can work based on voluntary contributions of reviewers. There is no rationale to write impartial and thorough evaluations. Consequently, there is no risk in submitting low-quality work by authors. As a result, scientists face a social dilemma: if everyone acts according to his or her own self-interest, low scientific quality is produced. Still, in practice, reviewers as well as authors invest high effort in reviews and submissions. We examine how the increased relevance of public good benefits (journal impact factor), the editorial policy of handling incoming reviews, and the acceptance decisions that take into account reputational information can help the evolution of high-quality contributions from authors. High effort from the side of reviewers is problematic even if authors cooperate: reviewers are still best off by producing low-quality reviews, which does not hinder scientific development, just adds random noise and unnecessary costs to it. We show with agent-based simulations that tacit agreements between authors that are based on reciprocity might decrease these costs, but does not result in superior scientific quality. Our study underlines why certain self-emerged current practices, such as the increased importance of journal metrics, the reputation-based selection of reviewers, and the reputation bias in acceptance work efficiently for scientific development. Our results find no answers, however, how the system of peer review with impartial and thorough evaluations could be sustainable jointly with rapid scientific development.Comment: Submitted to Scientometric

Open versus blind peer review: is anonymity better than transparency?

Peer review is widely accepted as essential to ensuring scientific quality in academic journals, yet little training is provided in the specifics of how to conduct peer review. In this article we describe the different forms of peer review, with a particular focus on the differences between single-blind, double-blind and open peer review, and the advantages and disadvantages of each. These illustrate some of the challenges facing the community of authors, editors, reviewers and readers in relation to the process of peer review. We also describe other forms of peer review, such as post-publication review, transferable review and collaborative review, and encourage clinicians and academics at all training stages to engage in the practice of peer review as part of continuing professional development

Perspectives on involvement in the peer-review process : surveys of patient and public reviewers at two journals

Objective In 2014/2015, The BMJ and Research Involvement and Engagement (RIE) became the first journals to routinely include patients and the public in the peer review process of journal articles. This survey explores the perspectives and early experiences of these reviewers. Design A cross-sectional survey. Setting and participants Patient and public reviewers for The BMJ and RIE who have been invited to review. Results The response rate was 69% (157/227) for those who had previously reviewed and 31% (67/217) for those who had not yet reviewed. Reviewers described being motivated to review by the opportunity to include the patient voice in the research process, influence the quality of the biomedical literature and ensure it meets the needs of patients. Of the 157 who had reviewed, 127 (81%) would recommend being a reviewer to other patients and carers. 144 (92%) thought more journals should adopt patient and public review. Few reviewers (16/224, 7%) reported concerns about doing open review. Annual acknowledgement on the journals’ websites was welcomed as was free access to journal information. Participants were keen to have access to more online resources and training to improve their reviewing skills. Suggestions on how to improve the reviewing experience included: allowing more time to review; better and more frequent communication; a more user-friendly process; improving guidance on how to review including videos; improving the matching of papers to reviewers’ experience; providing more varied sample reviews and brief feedback on the usefulness of reviews; developing a sense of community among reviewers; and publicising of the contribution that patient and public review brings. Conclusions Patient and public reviewers shared practical ideas to improve the reviewing experience and these will be reviewed to enhance the guidance and support given to them

Engaging undergraduate students in preprint peer review

Authentic assessment allows students to demonstrate knowledge and skills in real-world tasks. In research, peer review is one such task that researchers learn by doing, as they evaluate other researchers’ work. This means peer review could serve as an authentic assessment that engages students’ critical thinking skills in a process of active learning. In this study, we had students write peer reviews of preprints, scaffolded by a rubric. Agreement between the students and academics was reasonable, and active student involvement was high. The results suggest that use of peer review in undergraduate classes should be explored more. It likely facilitates students’ ability to evaluate the quality of scientific studies, encourages active learning about the scientific process and shows potential for contributing to publicly-available assessment of scientific studies

Engaging undergraduate students in preprint peer review

Authentic assessment allows students to demonstrate knowledge and skills in real-world tasks. In research, peer review is one such task that researchers learn by doing, as they evaluate other researchers’ work. This means peer review could serve as an authentic assessment that engages students’ critical thinking skills in a process of active learning. In this study, we had students write peer reviews of preprints, scaffolded by a rubric. Agreement between the students and academics was reasonable, and active student involvement was high. The results suggest that use of peer review in undergraduate classes should be explored more. It likely facilitates students’ ability to evaluate the quality of scientific studies, encourages active learning about the scientific process and shows potential for contributing to publicly-available assessment of scientific studies

Peer review: Objective screening or wishful thinking?

Despite the massive changes in academic publishing in recent years, one thing remains more or less constant: the disdain many academics feel for peer review, and perhaps for peer reviewers themselves. This is possibly the most contentious and secretive practice in our academic lives, reviled and tolerated in equal measure but rarely loved. Slow, biased, contradictory, hurtful or wilfully obtuse, reviewers come in for a lot of stick. But are the criticisms levelled at peer review merited or is it an effective means of separating the sweet from the sour? Here I argue that while there is a lot wrong with peer review, it serves an important function for the academic community and that ERPP teachers have a role to play in making it work