Evaluating Research Activity:Impact Factor vs. Research Factor

The Impact Factor (IF) “has moved ... from an obscure bibliometric indicator to become the chief quantitative measure of the quality of a journal, its research papers, the researchers who wrote those papers, and even the institution they work in” ([2], p. 1). However, the use of this index for evaluating individual scientists is dubious. The present work compares the ranking of research units generated by the Research Factor (RF) index with that associated with the popular IF. The former, originally introduced in [38], reflects article and book publications and a host of other activities categorized as coordination activities (e.g., conference organization, research group coordination), dissemination activities (e.g., conference and seminar presentations, participation in research group), editorial activities (e.g., journal editor, associate editor, referee) and functional activities (e.g., Head of Department). The main conclusion is that by replacing the IF with the RF in hiring, tenure decisions and awarding of grants would greatly increase the number of topics investigated and the number and quality of long run projects.scientific research assessment, Impact Factor, bibliometric indices, feasible Research Factor

Hit count estimate variability for website-specific queries in search engines: The case for rare disease association websites

"This article is (c) Emerald Group Publishing and permission has been granted for this version to appear here https://doi.org/10.1108/AJIM-10-2017-0226. Emerald does not grant permission for this article to be further copied/distributed or hosted elsewhere without the express permission from Emerald Group Publishing Limited"[EN] Purpose - The purpose of this paper is to determine the effect of the chosen search engine results page (SERP) on the website-specific hit count estimation indicator. Design/methodology/approach - A sample of 100 Spanish rare disease association websites is analysed, obtaining the website-specific hit count estimation for the first and last SERPs in two search engines (Google and Bing) at two different periods in time (2016 and 2017). Findings - It has been empirically demonstrated that there are differences between the number of hits returned on the first and last SERP in both Google and Bing. These differences are significant when they exceed a threshold value on the first SERP. Research limitations/implications - Future studies considering other samples, more SERPs and generating different queries other than website page count (ositeW) would be desirable to draw more general conclusions on the nature of quantitative data provided by general search engines. Practical implications - Selecting a wrong SERP to calculate some metrics (in this case, website-specific hit count estimation) might provide misleading results, comparisons and performance rankings. The empirical data suggest that the first SERP captures the differences between websites better because it has a greater discriminating power and is more appropriate for webometric longitudinal studies. Social implications - The findings allow improving future quantitative webometric analyses based on website-specific hit count estimation metrics in general search engines. Originality/value - The website-specific hit count estimation variability between SERPs has been empirically analysed, considering two different search engines (Google and Bing), a set of 100 websites focussed on a similar market (Spanish rare diseases associations), and two annual samples, making this study the most exhaustive on this issue to date.Font-Julian, CI.; Ontalba Ruipérez, JA.; Orduña Malea, E. (2018). Hit count estimate variability for website-specific queries in search engines: The case for rare disease association websites. Aslib Journal of Information Management. 70(2):192-213. https://doi.org/10.1108/AJIM-10-2017-0226S192213702Bar-Ilan, J. (2001). Scientometrics, 50(1), 7-32. doi:10.1023/a:1005682102768Bowler, L., Hong, W., & He, D. (2011). The visibility of health web portals for teens: a hyperlink analysis. Online Information Review, 35(3), 443-470. doi:10.1108/14684521111151469European Organization for Rare Diseases (2012), “What is a rare disease?”, available at: www.eurordis.org/content/what-rare-disease (accessed 10 January 2018).Forman, J., Taruscio, D., Llera, V. A., Barrera, L. A., Coté, T. R., … Edfjäll, C. (2012). The need for worldwide policy and action plans for rare diseases. Acta Paediatrica, 101(8), 805-807. doi:10.1111/j.1651-2227.2012.02705.xGao, Y., & Vaughan, L. (2005). Web hyperlink profiles of news sites. Aslib Proceedings, 57(5), 398-411. doi:10.1108/00012530510621851Gouveia, F. C., & Kurtenbach, E. (2009). Mapping the web relations of science centres and museums from Latin America. Scientometrics, 79(3), 491-505. doi:10.1007/s11192-007-1949-8Groselj, D. (2014). A webometric analysis of online health information: sponsorship, platform type and link structures. Online Information Review, 38(2), 209-231. doi:10.1108/oir-01-2013-0011Lewandowski, D. (2008). A three-year study on the freshness of web search engine databases. Journal of Information Science, 34(6), 817-831. doi:10.1177/0165551508089396Li, X. (2003). A review of the development and application of the Web impact factor. Online Information Review, 27(6), 407-417. doi:10.1108/14684520310510046Noruzi, A. (2006). The web impact factor: a critical review. The Electronic Library, 24(4), 490-500. doi:10.1108/02640470610689188Orduna-Malea, E. (2014), “Caracterización y rendimiento del sistema museístico de la comunidad valenciana a través de un análisis cibermétrico”, in Gimenez-Chornet, V. (Ed.), Gestión Cultural: Innovación y Tendencias, Tirant Lo Blanch, Valencia, pp. 13-43.Orduña-Malea, E., Delgado López-Cózar, E., Serrano-Cobos, J., & Romero, N. L. (2015). Disclosing the network structure of private companies on the web. Online Information Review, 39(3), 360-382. doi:10.1108/oir-11-2014-0282Park, H. W., Kim, C.-S., & Barnett, G. A. (2004). Socio-Communicational Structure among Political Actors on the Web in South Korea. New Media & Society, 6(3), 403-423. doi:10.1177/1461444804042522Rodríguez i Gairín, J. M. (1997). Valoración del impacto de la información en Internet: Altavista, el «Citation Index» de la red. Revista española de Documentación Científica, 20(2), 175-181. doi:10.3989/redc.1997.v20.i2.591Romero-Frías, E., & Vaughan, L. (2010). European political trends viewed through patterns of Web linking. Journal of the American Society for Information Science and Technology, 61(10), 2109-2121. doi:10.1002/asi.21375Satoh, K. and Yamana, H. (2012), “Hit count reliability: how much can we trust hit counts?”, in Sheng, Q.Z., Wang, G., Jensen, C.S. and Xu, G. (Eds), Asia-Pacific Web Conference, Springer, Berlin Heidelberg, April, pp. 751-758.Snyder, H., & Rosenbaum, H. (1999). Can search engines be used as tools for web‐link analysis? A critical view. Journal of Documentation, 55(4), 375-384. doi:10.1108/eum0000000007151Uyar, A. (2009). Investigation of the accuracy of search engine hit counts. Journal of Information Science, 35(4), 469-480. doi:10.1177/0165551509103598Vaughan, L., & Thelwall, M. (2004). Search engine coverage bias: evidence and possible causes. Information Processing & Management, 40(4), 693-707. doi:10.1016/s0306-4573(03)00063-3Vaughan, L., & Wu, G. (2004). Links to commercial websites as a source of business information. Scientometrics, 60(3), 487-496. doi:10.1023/b:scie.0000034389.14825.bcWilkinson, D., & Thelwall, M. (2013). Search markets and search results: The case of Bing. Library & Information Science Research, 35(4), 318-325. doi:10.1016/j.lisr.2013.04.00

L’ “R-Factor”: un nuovo modo di valutare la ricerca scientifica

As pointed out in Amin e Mabe (2000, p. 1), the journal impact factor (IF) “has moved in recent years from an obscure bibliometric indicator to become the chief quantitative measure of the quality of a journal, its research papers, the researchers who wrote those papers, and even the institution they work in.” However, the use of this index for evaluating individual scientists is dubious and may “skew the course of scientific research” (Monastersky, 2005, p, 1). Moreover the IF is, at most, able to measure only the quality of a very restricted range of research activities: namely, publishing journal articles. In the present work a new indicator of a researcher quality, named the Researcher Impact Factor (RF), is introduced. It is constructed as a function of the number and quality of publications (articles, books and working papers) and of the “other activities” usually associated with being a researcher (attending and/or organizing conferences, being the Editor, Associate Editor or referee for a journal, teaching or supervising at graduate level, coordinating research groups and so on). To show the characteristics of the new index, a numerical example is carried out to rank two hypothetical scientists. The main conclusion is that by replacing the IF with the RF in hiring, tenure decisions and awarding of grants would greatly increase the number of topics investigated and the number and quality of long run projects. The Excel spreadsheet used for the computations is available on demand from the authors.Impact factor, bibliometric indices, research evaluation, researcher impact factor

El empleo de la Webmetría para el análisis de los indicadores de desempeño y posición financiera de la empresa: un análisis exploratorio en diversos sectores económicos de los Estados Unidos

El desarrollo de Internet a lo largo de las dos últimas décadas ha dado lugar a unanueva disciplina que permite estudiar la Web desde un punto de vista cuantitativo: la webmetría.La aplicación de técnicas webmétricas al estudio de empresas apunta a la posible existencia derelaciones entre el número de enlaces que recibe un sitio web corporativo y determinadas variableseconómico-financieras. La incipiente evidencia encontrada se limita, en todo caso, a empresas pertenecientesa sectores tecnológicos que, por su propia naturaleza, cuentan con una fuerte presenciaen la Red. El presente trabajo, de carácter exploratorio, extiende el análisis a diversos sectoreseconómicos de los Estados Unidos, confirmando la existencia de una relación significativa entrelas variables expuestas. Esto abre nuevas posibilidades de investigación en el ámbito empresarial apartir del empleo de la información extraída de la estructura de enlaces de la Web

El empleo de la Webmetría para el análisis de los indicadores de desempeño y posición financiera de la empresa: un análisis exploratorio en diversos sectores económicos de los Estados Unidos

El desarrollo de Internet a lo largo de las dos últimas décadas ha dado lugar a unanueva disciplina que permite estudiar la Web desde un punto de vista cuantitativo: la webmetría.La aplicación de técnicas webmétricas al estudio de empresas apunta a la posible existencia derelaciones entre el número de enlaces que recibe un sitio web corporativo y determinadas variableseconómico-financieras. La incipiente evidencia encontrada se limita, en todo caso, a empresas pertenecientesa sectores tecnológicos que, por su propia naturaleza, cuentan con una fuerte presenciaen la Red. El presente trabajo, de carácter exploratorio, extiende el análisis a diversos sectoreseconómicos de los Estados Unidos, confirmando la existencia de una relación significativa entrelas variables expuestas. Esto abre nuevas posibilidades de investigación en el ámbito empresarial apartir del empleo de la información extraída de la estructura de enlaces de la Web