What Makes a Great Journal Great in the Sciences? Which Came First, the Chicken or the Egg?

The paper is concerned with analysing what makes a great journal great in the sciences, based on quantifiable Research Assessment Measures (RAM). Alternative RAM are discussed, with an emphasis on the Thomson Reuters ISI Web of Science database (hereafter ISI). Various ISI RAM that are calculated annually or updated daily are defined and analysed, including the classic 2-year impact factor (2YIF), 5-year impact factor (5YIF), Immediacy (or zero-year impact factor (0YIF)), Eigenfactor, Article Influence, C3PO (Citation Performance Per Paper Online), h-index, Zinfluence, PI-BETA (Papers Ignored - By Even The Authors), Impact Factor Inflation (IFI), and three new RAM, namely Historical Self-citation Threshold Approval Rating (H-STAR), 2 Year Self-citation Threshold Approval Rating (2Y-STAR), and Cited Article Influence (CAI). The RAM data are analysed for the 6 most highly cited journals in 20 highly-varied and well-known ISI categories in the sciences, where the journals are chosen on the basis of 2YIF. The application to these 20 ISI categories could be used as a template for other ISI categories in the sciences and social sciences, and as a benchmark for newer journals in a range of ISI disciplines. In addition to evaluating the 6 most highly cited journals in each of 20 ISI categories, the paper also highlights the similarities and differences in alternative RAM, finds that several RAM capture similar performance characteristics for the most highly cited scientific journals, determines that PI-BETA is not highly correlated with the other RAM, and hence conveys additional information regarding research performance. In order to provide a meta analysis summary of the RAM, which are predominantly ratios, harmonic mean rankings are presented of the 13 RAM for the 6 most highly cited journals in each of the 20 ISI categories. It is shown that emphasizing THE impact factor, specifically the 2-year impact factor, of a journal to the exclusion of other informative RAM can lead to a distorted evaluation of journal performance and influence on different disciplines, especially in view of inflated journal self citations.Research Assessment Measures (RAM); impact factors; Immediacy; Eigenfactor; Article Influence; Cited Article Influence; h-index; C3PO; Zinfluence; PI-BETA; IFI; H-STAR; 2Y-STAR

The New Zealand performance based research fund and its impact on publication activity in economics

New Zealand’s academic research assessment scheme, the Performance Based Research Fund (PBRF), was launched in 2002 with the stated objective of increasing research quality in the nation’s universities. Evaluation rounds were conducted in 2003, 2006 and 2012. In this paper, we employ 22 different journal weighting schemes to generate output estimates of refereed journal paper and page production over three six year periods (1994-1999; 2000-2005 and 2006-2011). These time periods reflect a pre-PBRF environment, a mixed assessment period, and a pure PBRF research environment, respectively. Our findings indicate that, on average, research productivity, defined in either paper or page terms, has increased since the introduction of the PBRF. However, this outcome is due to a major increase in the quantity of papers and pages produced per capita that has more than off-set a decline in the quality of published outputs since the introduction of the PBRF. In other words, our findings suggest that the PBRF has failed to achieve its stated goal of increasing average research quality, but it has resulted in substantial gains in productivity achieved via large increases in the quantity of refereed journal articles

Standardised library instruction assessment: an institution-specific approach

Introduction We explore the use of a psychometric model for locally-relevant, information literacy assessment, using an online tool for standardised assessment of student learning during discipline-based library instruction sessions. Method A quantitative approach to data collection and analysis was used, employing standardised multiple-choice survey questions followed by individual, cognitive interviews with undergraduate students. The assessment tool was administered to five general education psychology classes during library instruction sessions. AnalysisDescriptive statistics were generated by the assessment tool. Results. The assessment tool proved a feasible means of measuring student learning. While student scores improved on every survey question, there was uneven improvement from pre-test to post-test for different questions. Conclusion Student scores showed more improvement for some learning outcomes over others, thus, spending time on fewer concepts during instruction sessions would enable more reliable evaluation of student learning. We recommend using digital learning objects that address basic research skills to enhance library instruction programmes. Future studies will explore different applications of the assessment tool, provide more detailed statistical analysis of the data and shed additional light on the significance of overall scores

Great Expectatrics: Great Papers, Great Journals, Great Econometrics

The paper discusses alternative Research Assessment Measures (RAM), with an emphasis on the Thomson Reuters ISI Web of Science database (hereafter ISI). The various ISI RAM that are calculated annually or updated daily are defined and analysed, including the classic 2-year impact factor (2YIF), 5-year impact factor (5YIF), Immediacy (or zero-year impact factor (0YIF)), Eigenfactor score, Article Influence, C3PO (Citation Performance Per Paper Online), h-index, Zinfluence, and PI-BETA (Papers Ignored - By Even The Authors). The ISI RAM data are analysed for 8 leading econometrics journals and 4 leading statistics journals. The application to econometrics can be used as a template for other areas in economics, for other scientific disciplines, and as a benchmark for newer journals in a range of disciplines. In addition to evaluating high quality research in leading econometrics journals, the paper also compares econometrics and statistics, alternative RAM, highlights the similarities and differences in alternative RAM criteria, finds that several ISI RAM capture similar performance characteristics for the leading econometrics and statistics journals while the new PI-BETA criterion is not highly correlated with any of the other ISI RAM, and hence conveys additional information regarding ISI RAM, highlights major research areas in leading journals in econometrics, and discusses some likely future uses of RAM.Research assessment measures; impact factors; Immediacy; Eigenfactor score; Article influence; h-index; C3PO; Zinfluence; PI-BETA

Characterizing Elite Scholars of Library and Information Science: A Bibliometric Analysis

Publication metrics have become crucial to measure the productivity and impact of individuals and organizations for highlighting their influence upon the knowledge world. This study aims to present a bibliometric analysis of the top two percent highly cited Library and Information Science (LIS) researchers in the Stanford University list of the top two percent researchers of various disciplines published in 2021. We have used the bibliometric method to analyze 254 highly cited LIS researchers. The findings show male dominance and female underrepresentation in LIS research. Leydesdorff, Loet is the most highly cited researcher of the LIS field. The Amsterdam School of Communications Research, Netherlands, is the most highly cited organization of LIS based on the highly cited researchers affiliated with it. The majority of LIS researchers belong to USA and UK. There is a meager contribution of Asian, Australian, and African countries producing highly cited LIS researchers. Highly cited LIS researchers are more familiar with publishing research in information retrieval, information seeking, and bibliometrics. Positions of researchers at different ranks by different measures depict their productivity, impact, h- index, and a composite score. A strong correlation exists between total publications and total citations. The study, in its nature, is unique and will provide an insight for young and emerging researchers for selection of research direction, persons for research collaboration, and a motivation to invest more efforts for impactful research