Research assessment by percentile-based double rank analysis

In the double rank analysis of research publications, the local rank position of a country or institution publication is expressed as a function of the world rank position. Excluding some highly or lowly cited publications, the double rank plot fits well with a power law, which can be explained because citations for local and world publications follow lognormal distributions. We report here that the distribution of the number of country or institution publications in world percentiles is a double rank distribution that can be fitted to a power law. Only the data points in high percentiles deviate from it when the local and world $\mu$ parameters of the lognormal distributions are very different. The likelihood of publishing very highly cited papers can be calculated from the power law that can be fitted either to the upper tail of the citation distribution or to the percentile-based double rank distribution. The great advantage of the latter method is that it has universal application, because it is based on all publications and not just on highly cited publications. Furthermore, this method extends the application of the well-established percentile approach to very low percentiles where breakthroughs are reported but paper counts cannot be performed.Comment: A pdf file containing text, 9 figures and 4 tables. Accepted in Journal of Informetric

Information Security from a Scientometric Perspective

During recent years, Information societies are continually confronted with security threats and information vulnerabilities. Literature analysis of major disciplines is one of the key tools available to policymakers of research institutions and organizations. Scientometrics is the field of study which concerns itself with measuring and analyzing scientific literature. The purpose of the present study was to evaluate the research trends of information security in the Middle-East and the world from a scientometric perspective. Descriptive and analytical statistics were used in this study. The Scopus citation database was used to collect the publication data, as the largest abstract and citation database of peer-reviewed literature. Therefore, all documents related to "information security" were retrieved and analyzed. The results showed that the majority of scientific publications in the field of information security were produced in the United States of America and China. Among the Middle-Eastern countries, Iran ranked the first in terms of scientific publications in the field of information security while ranked 23th among the countries of the world. It is apparent that paying special attention to the field of information and data security in terms of international scientific collaborations, using knowledge and experiences of the leading, and supporting research and development in this field can improve the quality and quantity of scientific publications in this field. The expertise gathered during the process will prove invaluable and effective in confronting the security threats and vulnerabilities posed to information societies

A BIBLIOMETRIC ANALYSIS OF TWENTY-EIGHT YEARS ON ‘ASSESSING PRONUNCIATION IN ENGLISH AS A SECOND/FOREIGN LANGUAGE CLASSES’ (1993-2021)

This bibliometric study examines the characteristics of the overall research trends, patterns of productivity, and publications on “assessment in second language pronunciation”.  Bibliometric data were retrieved from Web of Science (WoS on 1 September 2021 and the results of the study reveal that the first publication appeared in 1993 and, during the period of 28 years, there have been 118 publications between 1993 and 2021 in total. It was found that studies in this field have increased in recent years. The publications include articles and proceeding papers written by 2.31 authors per publication. The most cited document received 139 citations. It was also discovered that the most frequently used word is intelligibility and the trending topic is pronunciation. As for the affiliations, the most productive university is Concordia University in Canada. In the following headings, detailed information is discussed in detail

Common bibliometric approaches fail to assess correctly the number of important scientific advances for most countries and institutions

Although not explicitly declared, most research rankings of countries and institutions are supposed to reveal their contribution to the advancement of knowledge. However, such advances are based on very highly cited publications with very low frequency, which can only very exceptionally be counted with statistical reliability. Percentile indicators enable calculations of the probability or frequency of such rare publications using counts of much more frequent publications; the general rule is that rankings based on the number of top 10% or 1% cited publications (Ptop 10%, Ptop 1%) will also be valid for the rare publications that push the boundaries of knowledge. Japan and its universities are exceptions, as their frequent Nobel Prizes contradicts their low Ptop 10% and Ptop 1%. We explain that this occurs because, in single research fields, the singularity of percentile indicators holds only for research groups that are homogeneous in their aims and efficiency. Correct calculations for ranking countries and institutions should add the results of their homogeneous groups, instead of considering all publications as a single set. Although based on Japan, our findings have a general character. Common predictions of scientific advances based on Ptop 10% might be severalfold lower than correct calculations.Comment: 30 pages, tables and figures embedded in a single pdf fil

Rank analysis of most cited publications, a new approach for research assessments

Citation metrics are the best tools for research assessments. However, current metrics are misleading in research systems that pursue simultaneously different goals, such as the advance of science and incremental innovations, because their publications have different citation distributions. We estimate the contribution to the progress of knowledge by studying only a limited number of the most cited papers, which are dominated by publications pursuing this progress. To field-normalize the metrics, we substitute the number of citations by the rank position of papers from one country in the global list of papers. Using synthetic series of lognormally distributed numbers, we developed the Rk-index, which is calculated from the global ranks of the 10 highest numbers in each series, and demonstrate its equivalence to the number of papers in top percentiles, P top 0.1% and P top 0.01% . In real cases, the Rk-index is simple and easy to calculate, and evaluates the contribution to the progress of knowledge much better than commonly used metrics. Although further research is needed, rank analysis of the most cited papers is a promising approach for research evaluation. It is also demonstrated that, for this purpose, domestic and collaborative papers should be studied independently.Comment: One PDF file, including figures and tables (31 pages

Uncertain research country rankings. Should we continue producing uncertain rankings?

Citation based country rankings consistently categorize Japan as a developing country, even in those from the most reputed institutions. This categorization challenges the credibility of such rankings, considering Japan elevated scientific standing. In most cases, these rankings use percentile indicators and are accurate if country citations fit an ideal model of distribution, but they can be misleading in cases of deviations. The ideal model implies a lognormal citation distribution and a power law citation based double rank: in the global and country lists. This report conducts a systematic examination of deviations from the ideal model and their consequential impact on evaluations. The study evaluates six selected countries across three scientifically relevant topics and utilizes Leiden Ranking assessments of over 300 universities. The findings reveal three types of deviations from the lognormal citation distribution: i deviations in the extreme upper tail; ii inflated lower tails; and iii deflated lower part of the distributions. These deviations stem from structural differences among research systems that are prevalent and have the potential to mislead evaluations across all research levels. Consequently, reliable evaluations must consider these deviations. Otherwise, while some countries and institutions will be correctly evaluated, failure to identify deviations in each specific country or institution will render uncertain evaluations. For reliable assessments, future research evaluations of countries and institutions must identify deviations from the ideal model.Comment: 29 pages, 6 figures, 5 table

Information Security from a Scientometric Perspective

During recent years, Information societies are continually confronted with security threats and information vulnerabilities. Literature analysis of major disciplines is one of the key tools available to policymakers of research institutions and organizations. Scientometrics is the field of study which concerns itself with measuring and analyzing scientific literature. The purpose of the present study was to evaluate the research trends of information security in the Middle-East and the world from a scientometric perspective. Descriptive and analytical statistics were used in this study. The Scopus citation database was used to collect the publication data, as the largest abstract and citation database of peer-reviewed literature. Therefore, all documents related to "information security" were retrieved and analyzed. The results showed that the majority of scientific publications in the field of information security were produced in the United States of America and China. Among the Middle-Eastern countries, Iran ranked the first in terms of scientific publications in the field of information security while ranked 23th among the countries of the world. It is apparent that paying special attention to the field of information and data security in terms of international scientific collaborations, using knowledge and experiences of the leading, and supporting research and development in this field can improve the quality and quantity of scientific publications in this field. The expertise gathered during the process will prove invaluable and effective in confronting the security threats and vulnerabilities posed to information societies