Peer review quality and transparency of the peer-review process in open access and subscription journals

BACKGROUND:Recent controversies highlighting substandard peer review in Open Access (OA) and traditional (subscription) journals have increased the need for authors, funders, publishers, and institutions to assure quality of peer-review in academic journals. I propose that transparency of the peer-review process may be seen as an indicator of the quality of peer-review, and develop and validate a tool enabling different stakeholders to assess transparency of the peer-review process. METHODS AND FINDINGS:Based on editorial guidelines and best practices, I developed a 14-item tool to rate transparency of the peer-review process on the basis of journals' websites. In Study 1, a random sample of 231 authors of papers in 92 subscription journals in different fields rated transparency of the journals that published their work. Authors' ratings of the transparency were positively associated with quality of the peer-review process but unrelated to journal's impact factors. In Study 2, 20 experts on OA publishing assessed the transparency of established (non-OA) journals, OA journals categorized as being published by potential predatory publishers, and journals from the Directory of Open Access Journals (DOAJ). Results show high reliability across items (α = .91) and sufficient reliability across raters. Ratings differentiated the three types of journals well. In Study 3, academic librarians rated a random sample of 140 DOAJ journals and another 54 journals that had received a hoax paper written by Bohannon to test peer-review quality. Journals with higher transparency ratings were less likely to accept the flawed paper and showed higher impact as measured by the h5 index from Google Scholar. CONCLUSIONS:The tool to assess transparency of the peer-review process at academic journals shows promising reliability and validity. The transparency of the peer-review process can be seen as an indicator of peer-review quality allowing the tool to be used to predict academic quality in new journals

Are Mendeley Reader Counts Useful Impact Indicators in all Fields?

Reader counts from the social reference sharing site Mendeley are known to be valuable for early research evaluation. They have strong correlations with citation counts for journal articles but appear about a year before them. There are disciplinary differences in the value of Mendeley reader counts but systematic evidence is needed at the level of narrow fields to reveal its extent. In response, this article compares Mendeley reader counts with Scopus citation counts for journal articles from 2012 in 325 narrow Scopus fields. Despite strong positive correlations in most fields, averaging 0.671, the correlations in some fields are as weak as 0.255. Technical reasons explain most weaker correlations, suggesting that the underlying relationship is almost always strong. The exceptions are caused by unusually high educational or professional use or topics of interest within countries that avoid Mendeley. The findings suggest that if care is taken then Mendeley reader counts can be used for early citation impact evidence in almost all fields and for related impact in some of the remainder. As an additional application of the results, cross-checking with Mendeley data can be used to identify indexing anomalies in citation databases

Are citations from clinical trials evidence of higher impact research? An analysis of ClinicalTrials.gov

An important way in which medical research can translate into improved health outcomes is by motivating or influencing clinical trials that eventually lead to changes in clinical practice. Citations from clinical trials records to academic research may therefore serve as an early warning of the likely future influence of the cited articles. This paper partially assesses this hypothesis by testing whether prior articles referenced in ClinicalTrials.gov records are more highly cited than average for the publishing journal. The results from four high profile general medical journals support the hypothesis, although there may not be a cause-and effect relationship. Nevertheless, it is reasonable for researchers to use citations to their work from clinical trials records as partial evidence of the possible long-term impact of their research

Benchmarking the scientific output of industrial wastewater research in Arab world by utilizing bibliometric techniques

Rapid population growth, worsening of the climate, and severity of freshwater scarcity are global challenges. In Arab world countries, where water resources are becoming increasingly scarce, the recycling of industrial wastewater could improve the efficiency of freshwater use. The benchmarking of scientific output of industrial wastewater research in the Arab world is an initiative that could support in shaping up and improving future research activities. This study assesses the scientific output of industrial wastewater research in the Arab world. A total of 2032 documents related to industrial wastewater were retrieved from 152 journals indexed in the Scopus databases; this represents 3.6 % of the global research output. The h-index of the retrieved documents was 70. The total number of citations, at the time of data analysis, was 34,296 with an average citation of 16.88 per document. Egypt, with a total publications of 655 (32.2 %), was ranked the first among the Arab countries followed by Saudi Arabia 300 (14.7 %) and Tunisia 297 (14.6 %). Egypt also had the highest h-index, assumed with Saudi Arabia, the first place in collaboration with other countries. Seven hundred fifteen (35.2 %) documents with 66 countries in Arab/non-Arab country collaborations were identified. Arab researchers collaborated mostly with researchers from France 239 (11.7 %), followed by the USA 127 (6.2 %). The top active journal was Desalination 126 (6.2 %), and the most productive institution was the National Research Center, Egypt 169 (8.3 %), followed by the King Abdul-Aziz University, Saudi Arabia 75 (3.7 %). Environmental Science was the most prevalent field of interest 930 (45.8 %). Despite the promising indicators, there is a need to close the gap in research between the Arab world and the other nations. Optimizing the investments and developing regional experiences are key factors to promote the scientific research

Selected Theses on Science

peer reviewedThe purpose of Science is to achieve the truth on the way to a new knowledge. The truth, as Immanuel Kant wrote, is the correspondence of knowledge with its object. However, the key question is how to “find a universal and true criterion of the truth of all knowledge”? The contribution of the fundamental sciences is extremely important. And here, in my opinion, there appears a modern paradox which has globally changed the public consciousness. On the one hand, the fundamental science went into the status of the labor forces and, on the other hand, modern production, demanding “the implementation of scientific research and scientific approach, began increasingly resemble to science.” In the process of production—which creates the product of labor including both material goods and services in the case of material production and a new knowledge as in the case of science—the labor forces enter into industrial relations. If any scientific work as an object, an element of the external world that we aim to contemplate the work, as well as a phenomenon, and develop its conceptual representation as well as about the phenomenon that it is modeling. Hence, the closer to the actual simulated phenomenon to the studied one, the closer this work to the truth. I assume my viewpoint is quite clear, even without mentioning Goethe: “It is a shame that the truth is so simple.”. © 2018, Springer International Publishing AG, part of Springer Nature

Do social sciences and humanities behave like life and hard sciences?

The quantitative evaluation of Social Science and Humanities (SSH) and the investigation of the existing similarities between SSH and Life and Hard Sciences (LHS) represent the forefront of scientometrics research. We analyse the scientific production of the universe of Italian academic scholars , over a 10-year period across 2002–2012, from a national database built by the Italian National Agency for the Evaluation of Universities and Research Institutes. We demonstrate that all Italian scholars of SSH and LHS are equals, as far as their publishing habits. They share the same general law, which is a lognormal. At the same time, however, they are different, because we measured their scientific production with different indicators required by the Italian law; we eliminated the “silent” scholars and obtained different scaling values—proxy of their productivity rates. Our findings may be useful to further develop indirect quali–quantitative comparative analysis across heterogeneous disciplines and, more broadly, to investigate on the generative mechanisms behind the observed empirical regularities. © 2017, Akadémiai Kiadó, Budapest, Hungary