What indicators matter? The Analysis of Perception toward Research Assessment Indicators and Leiden Manifesto- The Case Study of Taiwan

Abstract

This study aims to investigate the Taiwanese researchers' awareness toward bibliometric indicators and the principles from Leiden Manifesto. The online survey was conducted and obtained a total of 417 valid responses. The results show that evoking the right concept of use of bibliometric indicators and research evaluation has a long way to go. The lack of recognition of bibliometric indicators exists in Taiwan academia. Generally speaking, researchers may hear of the certain indicators, but they are not familiar its definition and calculation process. Only JIF and h-index are considered as well-known indicators. Compared to San Francisco Declaration on Research Assessment (DORA), Leiden Manifesto has lower visibility in Taiwan, hence, this study also aims to promote the right concepts of research evaluation by conducting questionnaire survey. The analysis results suggest that the ten principles can be considered the universal guideline in research evaluation since most of Taiwanese researchers agree the contents of ten principles. Especially for the principle 6 “Account for variation by field in publication and citation practices” has less room of opinion diversity. However, it is interesting to compare the result of recognition of relative citation ratio, only few researchers have fully understood the definition. This result indicates that scientometricians should need to make more effort to disseminate the concept of field-normalization in bibliometric indicators. The researchers do have understanding about the importance of comparison on the same basis, at the meantime, they may use the inappropriate indicators just because lack of enough knowledge on the variety of indicators. Hence, it is important to initiate that education of informetrics to all of the stakeholders in research evaluation so that the misuse and abuse of bibliometric indicators may possibly do not happen again, and the bibliometric analysis is able to turn to contextualization-based analysis in the future