The emergence of a field: a network analysis of research on peer review

This article provides a quantitative analysis of peer review as an emerging field of research by revealing patterns and connections between authors, fields and journals from 1950 to 2016. By collecting all available sources from Web of Science, we built a dataset that included approximately 23,000 indexed records and reconstructed collaboration and citation networks over time. This allowed us to trace the emergence and evolution of this field of research by identifying relevant authors, publications and journals and revealing important development stages. Results showed that while the term “peer review” itself was relatively unknown before 1970 (“referee” was more frequently used), publications on peer review significantly grew especially after 1990. We found that the field was marked by three development stages: (1) before 1982, in which most influential studies were made by social scientists; (2) from 1983 to 2002, in which research was dominated by biomedical journals, and (3) from 2003 to 2016, in which specialised journals on science studies, such as Scientometrics, gained momentum frequently publishing research on peer review and so becoming the most influential outlets. The evolution of citation networks revealed a body of 47 publications that form the main path of the field, i.e., cited sources in all the most influential publications. They could be viewed as the main corpus of knowledge for any newcomer in the fiel

Determinants of quality of research environment: An assessment of the environment submissions in the UK’s Research Excellence Framework in 2014

One of the assessed research elements in the UK’s Research Excellence Framework (REF) exercise in 2014 was the research environment. The quality of the research environment was assessed by expert peer reviewers who were given a set of quantitative factors to support their decision making. However, there is no systematic procedure to integrate this quantitative information into the evaluation process. This article evaluates the relevance of quantitative factors in explaining the assessed quality of the research environment. Findings suggest submitting units with high external research income generation tend to have a better research environment evaluation in almost all the assessed subject areas. The importance given by reviewers to similar quantitative factors was distinctively different in two units of assessment (UoA) in which the evaluation criteria were the same, which highlights the internal inconsistency of the peer review evaluation. Our findings also confirm the existence of the ‘halo effect’ in some UoA where submitting units that belong to the Russell group and have sub-panel members in the REF exercise obtained higher scores even after controlling for the assessed quantitative factors

Tieteellinen kirjoittaminen tutkijaryhmänä

HYMY-tutkijaryhmä kertoo artikkelissaan, miten kilpailun, kiireen ja muiden institutionaalisten paineiden leimaamassa akateemisessa maailmassa on mahdollista toteuttaa akateemiseen vapauteen kuuluvia ihanteita tehdä työtä. Ryhmä on löytänyt tavan tehdä työtä ryhmänä ilman aikapaineita, yhteisiä tavoitteita asettaen ja yhteisistä arvoista ponnistaen. Akateeminen vapaus toteutuu, kun ryhmä tekee työtään kaikkia jäseniä kiinnostaviin tutkimusaiheisiin paneutuen ilman ulkopuolista rahoitusta, mutta kollektiivisia tavoitteita asettaen. Julkaisuprosessin aikana ryhmä noudattaa vakiintuneita institutionaalisia käytänteitä

Kako recenzirati naučni rad

A review process is a key factor which ensures reliable and accurate presentation of new, useful, and original scientific knowledge to the public. Despite of many shortcomings which this evaluation of scientific work has, it is still an indispensable part of the process of scientific publishing. Different types of reviews have emerged throughout history, but the essence of the process itself has remained unchanged: before publishing, scientific results are subjected to unbiased, competent, and reliable assessment of their validity and originality. Unfortunately, neither worldwide, nor in our country, there is systematic and institutionalized education for performing such responsible task. The knowledge on review process is taught by senior colleagues, or is acquired through personal experience over time. As each young scientist is a potential reviewer, this article Is primarily intended for young people, as a manual, instruction on how to review a scientific paper and what should be kept in mind when the review report is written. After the analysis of the review process, ethical principles to which a reviewer should adhere to are highlighted, and finally, article intends to answer a question: How to review a scientific paper critically, correctly, and objectively? At the end, it is suggested how the review report should look like.Postupak recenziranja je ključni element koji obezbeđuje pouzdano i tačno prezentovanje novog, korisnog i originalnog naučnog saznanja javnosti. I pored mnogih nedostataka koje ovakvo vrednovanje naučnog rada ima, ono je do danas nezamenjivi deo procesa publikovanja rezultata naučnih istraživanja. Tokom istorije razvijale su se različite vrste recenziranja, ali suština samog procesa je ostala nepromenjena: pre publikovanja rezultati se podvrgavaju nepristrasnoj, kompetentnoj i pouzdanoj oceni valjanosti, vrednosti i originalnosti. Nažalost, ni u svetu, a pogotovo kod nas, skoro da ne postoji sistematsko i institucionalizovano obrazovanje za obavljanje ovog odgovornog zadatka. Znanje i praksa se preuzimaju od starijih kolega, ili se stiču vremenom. Kako je svaki mladi naučnik potencijalni recenzent, ovaj tekst je namenjen pre svega mladima, kao priručnik, uputstvo kako recenzirati naučni rad i šta sve treba imati u vidu kada se piše recenzentski izveštaj. Nakon analize procesa recenziranja, u tekstu su istaknuti etički principi kojih se recenzent treba pridržavati, a zatim je pokušano dati odgovor na pitanje: Kako kritički, korektno i objektivno recenzirati naučni rad? Na kraju, sugerisano je kako recenzentski izveštaj treba da izgleda

Academics' attitudes towards peer review in scholarly journals and the effect of role and discipline

This research contributes to the knowledge on academics’ attitudes towards peer review, through an international and inter-disciplinary survey of academics, which profiles academics’ views on the value of peer review, its benefits and the prevalence of unethical practices. Generally, academics regarded peer review as beneficial to improving their article and felt that peer review contributed significantly to the effectiveness of scholarly communication. Academics agreed that peer review could improve the readability and quality of the published paper, as well as check for accuracy, appropriate methodology, novelty and relevance to the journal. There are significant differences in the views of respondents on the basis of role, with those involved as reviewers and editors being less positive about peer review than authors. In addition, there is evidence of some disciplinary differences in views on the benefits of peer review