Influence of the Academic Library on US University Reputation: A Webometric Approach

A previous study conducted through a survey of academic libraries at 100 US universities with the highest total expenditures on academic libraries according to data presented by the National Center for Education Statistics (NCES). The results pointed out an unexpectedly weak correlation among web variables, concluding that the complex online structure of US academic libraries was the main driver of this effect. The present study replicates this research applying the same web indicators but at the university level, to check whether the weak compactness among web indicators persists. Additionally, the percentage (in terms of web data) of academic libraries at universities is analyzed. Finally, the correlation among web and economic indicators (research expenditures, student population, and reputational rank position) for universities is calculated to check for a possible relationship. Results confirm a strong correlation among university web indicators. Otherwise, the strength of academic libraries at universities is moderate in terms of page count, but weak in terms of visits. Finally, the correlation among university web indicators and research expenditures depends on student population.Orduña Malea, E.; Regazzi, JJ. (2013). Influence of the Academic Library on US University Reputation: A Webometric Approach. Technologies. 1(2):26-43. doi:10.3390/technologies1020026S264312Centre for Higher Education (CHE)http://www.che.dehttp://tools.macleans.ca/ranking2013/selectindicators.aspxhttp://www.princetonreview.com/college-rankings.aspxRegazzi, J. J. (2012). Constrained? An Analysis of U.S. Academic Library Shifts in Spending, Staffing, and Utilization, 1998–2008. College & Research Libraries, 73(5), 449-468. doi:10.5860/crl-260Regazzi, J. J. (2012). Comparing Academic Library Spending with Public Libraries, Public K-12 Schools, Higher Education Public Institutions, and Public Hospitals Between 1998–2008. The Journal of Academic Librarianship, 38(4), 205-216. doi:10.1016/j.acalib.2012.04.003Oppenheim, C., & Stuart, D. (2004). Is there a correlation between investment in an academic library and a higher education institution’s ratings in the Research Assessment Exercise? Aslib Proceedings, 56(3), 156-165. doi:10.1108/00012530410699578Noh, Y. (2012). The impact of university library resources on university research achievement outputs. Aslib Proceedings, 64(2), 109-133. doi:10.1108/00012531211215150http://www.webometrics.infoAguillo, I. F., Ortega, J. L., & Fernández, M. (2008). Webometric Ranking of World Universities: Introduction, Methodology, and Future Developments. Higher Education in Europe, 33(2-3), 233-244. doi:10.1080/03797720802254031Ortega, J. L., & Aguillo, I. F. (2009). Mapping world-class universities on the web. Information Processing & Management, 45(2), 272-279. doi:10.1016/j.ipm.2008.10.001Thelwall, M., & Zuccala, A. (2008). A university-centred European Union link analysis. Scientometrics, 75(3), 407-420. doi:10.1007/s11192-007-1831-8Qiu, J., Chen, J., & Wang, Z. (2004). An analysis of backlink counts and Web Impact Factorsfor Chinese university websites. Scientometrics, 60(3), 463-473. doi:10.1023/b:scie.0000034387.76981.83Smith, A., & Thelwall, M. (2002). Scientometrics, 54(3), 363-380. doi:10.1023/a:1016030415822Park, H. W., & Thelwall, M. (2006). Web-science communication in the age of globalization. New Media & Society, 8(4), 629-650. doi:10.1177/1461444806065660Barnett, G. A., Park, H. W., Jiang, K., Tang, C., & Aguillo, I. F. (2013). A multi-level network analysis of web-citations among the world’s universities. Scientometrics, 99(1), 5-26. doi:10.1007/s11192-013-1070-0Aguillo, I. F., Granadino, B., Ortega, J. L., & Prieto, J. A. (2006). Scientific research activity and communication measured with cybermetrics indicators. Journal of the American Society for Information Science and Technology, 57(10), 1296-1302. doi:10.1002/asi.20433Lee, M., & Park, H. W. (2011). Exploring the web visibility of world-class universities. Scientometrics, 90(1), 201-218. doi:10.1007/s11192-011-0515-6Arakaki, M., & Willett, P. (2008). Webometric analysis of departments of librarianship and information science: a follow-up study. Journal of Information Science, 35(2), 143-152. doi:10.1177/0165551508094051Tang, R., & Thelwall, M. (2008). A Hyperlink Analysis of U.S. Public and Academic Libraries’ Web Sites. The Library Quarterly, 78(4), 419-435. doi:10.1086/591179Orduña-Malea, E., & Regazzi, J. J. (2013). U.S. academic libraries: understanding their web presence and their relationship with economic indicators. Scientometrics, 98(1), 315-336. doi:10.1007/s11192-013-1001-0http://colleges.usnews.rankingsandreviews.com/best-colleges/rankings/national-universitieshttp://www.forbes.com/top-collegesAguillo, I. (2009). Measuring the institution’s footprint in the web. Library Hi Tech, 27(4), 540-556. doi:10.1108/073788309http://www.majesticseo.comhttp://www.opensiteexplorer.orghttps://ahrefs.comThelwall, M., & Sud, P. (2011). A comparison of methods for collecting web citation data for academic organizations. Journal of the American Society for Information Science and Technology, 62(8), 1488-1497. doi:10.1002/asi.21571http://www.alexa.com/http://digibug.ugr.es/handle/10481/2375

Are web mentions accurate substitutes for inlinks for Spanish universities?

This article is (c) Emerald Group Publishing and permission has been granted for this version to appear here. Emerald does not grant permission for this article to be further copied/distributed or hosted elsewhere without the express permission from Emerald Group Publishing Limitedurpose – Title and URL mentions have recently been proposed as web visibility indicators instead of inlink counts. The objective of this study is to determine the accuracy of these alternative web mention indicators in the Spanish academic system, taking into account their complexity (multi-domains) and diversity (different official languages). Design/methodology/approach – Inlinks, title and URL mentions from 76 Spanish universities were manually extracted from the main search engines (Google, Google Scholar, Yahoo!, Bing and Exalead). Several statistical methods, such as correlation, difference tests and regression models, were used. Findings – Web mentions, despite some limitations, can be used as substitutes for inlinks in the Spanish academic system, although these indicators are more likely to be influenced by the environment (language, web domain policy, etc.) than inlinks. Research limitations/implications – Title mentions provide unstable results caused by the multiple name variants which an institution can present (such as acronyms and other language versions). URL mentions are more stable, but they may present atypical points due to some shortcomings, the effect of which is that URL mentions do not have the same meaning as inlinks. Practical implications – Web mentions should be used with caution and after a cleaning-up process. Moreover, these counts do not necessarily signify connectivity, so their use in global web analysis should be limited. Originality/value – Web mentions have previously been used in some specific academic systems (US, UK and China), but this study analyses, in depth and for the first time, an entire non-English speaking European country (Spain), with complex academic web behaviour, which helps to better explain previous web mention results.Ortega, JL.; Orduña Malea, E.; Aguillo, IF. (2014). Are web mentions accurate substitutes for inlinks for Spanish universities?. Online Information Review. 38(1):59-77. doi:10.1108/OIR-10-2012-0189S5977381Adecannby, J. (2011), “Web link analysis of interrelationship between top ten African universities and world universities”, Annals of Library and Information Studies, Vol. 58 No. 2, pp. 128-138.Aguillo, I. (2009). Measuring the institution’s footprint in the web. Library Hi Tech, 27(4), 540-556. doi:10.1108/073788309Aguillo, I.F. , Ortega, J.L. and Fernández, M. (2008), “Webometric ranking of world universities: introduction, methodology, and future developments”, Higher Education in Europe, Vol. 33 Nos 2/3, pp. 234-244.Aguillo, I. F., Granadino, B., Ortega, J. L., & Prieto, J. A. (2006). Scientific research activity and communication measured with cybermetrics indicators. Journal of the American Society for Information Science and Technology, 57(10), 1296-1302. doi:10.1002/asi.20433Barabási, A.-L., & Albert, R. (1999). Emergence of Scaling in Random Networks. Science, 286(5439), 509-512. doi:10.1126/science.286.5439.509Bar-Ilan, J. (2005). The use of web search engines in information science research. Annual Review of Information Science and Technology, 38(1), 231-288. doi:10.1002/aris.1440380106Bar-Ilan, J. (2004). A microscopic link analysis of academic institutions within a country - the case of Israel. Scientometrics, 59(3), 391-403. doi:10.1023/b:scie.0000018540.33706.c1Bar-Ilan, J. (2005). What do we know about links and linking? A framework for studying links in academic environments. Information Processing & Management, 41(4), 973-986. doi:10.1016/j.ipm.2004.02.005Björneborn, L., & Ingwersen, P. (2004). Toward a basic framework for webometrics. Journal of the American Society for Information Science and Technology, 55(14), 1216-1227. doi:10.1002/asi.20077Bland, J.M. and Altman, D.G. (1996), “Transforming data”, British Medical Journal, Vol. 312 No. 7033, p.Cronin, B., Snyder, H. W., Rosenbaum, H., Martinson, A., & Callahan, E. (1998). Invoked on the Web. Journal of the American Society for Information Science, 49(14), 1319-1328. doi:10.1002/(sici)1097-4571(1998)49:143.0.co;2-wFriedman, M. (1937). The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance. Journal of the American Statistical Association, 32(200), 675-701. doi:10.1080/01621459.1937.10503522Harries, G., Wilkinson, D., Price, L., Fairclough, R., & Thelwall, M. (2004). Hyperlinks as a data source for science mapping. Journal of Information Science, 30(5), 436-447. doi:10.1177/0165551504046736Heimeriks, G. and Van den Besselaar, P. (2006), “Analyzing hyperlink networks: the meaning of hyperlink based indicators of knowledge”, Cybermetrics, Vol. 10, available at: http://cybermetrics.cindoc.csic.es/articles/v10i1p1.pdf (accessed 10 July 2013).Heimeriks, G., Hörlesberger, M., & Van den Besselaar, P. (2003). Scientometrics, 58(2), 391-413. doi:10.1023/a:1026296812830Kousha, K. and Horri, A. (2004), “The relationship between scholarly publishing and the counts of academic inlinks to Iranian university web sites: exploring academic link creation motivations”, Journal of Information Management and Scientometrics, Vol. 1 No. 2, pp. 13-22.Kousha, K., & Thelwall, M. (2009). Google book search: Citation analysis for social science and the humanities. Journal of the American Society for Information Science and Technology, 60(8), 1537-1549. doi:10.1002/asi.21085Kretschmer, H., & Aguillo, I. F. (2004). Visibility of collaboration on the Web. Scientometrics, 61(3), 405-426. doi:10.1023/b:scie.0000045118.68430.fdOrduña-Malea, E. (2012), “Fuentes de enlaces web para análisis cibermétricos (2012)”, Anuario Thinkepi, Vol. 6 No. 1, pp. 276-280.Orduña-Malea, E. (2013), “Espacio universitario español en la Web (2010): estudio descriptivo de instituciones y productos académicos a través del análisis de subdominios y subdirectorios”, Revista Española de Documentación Científica, Vol. 36 No. 3.Orduña-Malea, E., & Ontalba-Ruipérez, J.-A. (2012). Proposal for a multilevel university cybermetric analysis model. Scientometrics, 95(3), 863-884. doi:10.1007/s11192-012-0868-5Orduña-Malea, E., Serrano-Cobos, J., Ontalba-Ruipérez, J. A., & Lloret-Romero, N. (2010). Presencia y visibilidad web de las universidades públicas españolas. Revista española de Documentación Científica, 33(2), 246-278. doi:10.3989/redc.2010.2.740Ortega, J. L., & Aguillo, I. F. (2008). Visualization of the Nordic academic web: Link analysis using social network tools. Information Processing & Management, 44(4), 1624-1633. doi:10.1016/j.ipm.2007.09.010Ortega, J. L., & Aguillo, I. F. (2009). Análisis estructural de la web académica iberoamericana. Revista española de Documentación Científica, 32(3), 51-65. doi:10.3989/redc.2009.3.689Ortega, J. L., Aguillo, I., Cothey, V., & Scharnhorst, A. (2007). Maps of the academic web in the European Higher Education Area — an exploration of visual web indicators. Scientometrics, 74(2), 295-308. doi:10.1007/s11192-008-0218-9Qiu, J., Chen, J., & Wang, Z. (2004). An analysis of backlink counts and Web Impact Factorsfor Chinese university websites. Scientometrics, 60(3), 463-473. doi:10.1023/b:scie.0000034387.76981.83Seeber, M., Lepori, B., Lomi, A., Aguillo, I., & Barberio, V. (2012). Factors affecting web links between European higher education institutions. Journal of Informetrics, 6(3), 435-447. doi:10.1016/j.joi.2012.03.001Seidman, E. (2007), “We are flattered, but …”, Bing Community, available at: www.bing.com/community/site_blogs/b/search/archive/2007/03/28/we-are-flattered-but.aspx (accessed 20 October 2012).Smith, A.G. (1999), “A tale of two web spaces: comparing sites using web impact factors”, Journal of Documentation, Vol. 55 No. 5, pp. 577-592.Smith, A., & Thelwall, M. (2002). Scientometrics, 54(3), 363-380. doi:10.1023/a:1016030415822Stuart, D., & Thelwall, M. (2006). Investigating triple helix relationships using URL citations: a case study of the UK West Midlands automobile industry. Research Evaluation, 15(2), 97-106. doi:10.3152/147154406781775968Thelwall, M. (2001). Extracting macroscopic information from Web links. Journal of the American Society for Information Science and Technology, 52(13), 1157-1168. doi:10.1002/asi.1182Thelwall, M. (2002). An initial exploration of the link relationship between UK university Web sites. Aslib Proceedings, 54(2), 118-126. doi:10.1108/00012530210435248Thelwall, M. and Aguillo, I.F. (2003), “La salud de las web universitarias españolas”, Revista Española de Documentación Científica, Vol. 26 No. 3, pp. 291-305.Thelwall, M., & Kousha, K. (2008). Online presentations as a source of scientific impact? An analysis of PowerPoint files citing academic journals. Journal of the American Society for Information Science and Technology, 59(5), 805-815. doi:10.1002/asi.20803Thelwall, M., & Sud, P. (2011). A comparison of methods for collecting web citation data for academic organizations. Journal of the American Society for Information Science and Technology, 62(8), 1488-1497. doi:10.1002/asi.21571Thelwall, M., & Sud, P. (2012). Webometric research with the Bing Search API 2.0. Journal of Informetrics, 6(1), 44-52. doi:10.1016/j.joi.2011.10.002Thelwall, M., & Zuccala, A. (2008). A university-centred European Union link analysis. Scientometrics, 75(3), 407-420. doi:10.1007/s11192-007-1831-8Thelwall, M., Tang, R., & Price, L. (2003). Scientometrics, 56(3), 417-432. doi:10.1023/a:1022387105904Thelwall, M., Binns, R., Harries, G., Page-Kennedy, T., Price, L., & Wilkinson, D. (2002). Scientometrics, 53(1), 95-111. doi:10.1023/a:1014836021080Vaughan, L. (2012). An Alternative Data Source for Web Hyperlink Analysis: «Sites Linking In» at Alexa Internet. Collnet Journal of Scientometrics and Information Management, 6(1), 31-42. doi:10.1080/09737766.2012.10700922Vaughan, L., & Romero-Frías, E. (2012). Exploring Web keyword analysis as an alternative to link analysis: a multi-industry case. Scientometrics, 93(1), 217-232. doi:10.1007/s11192-012-0640-xVaughan, L., & Shaw, D. (2003). Bibliographic and Web citations: What is the difference? Journal of the American Society for Information Science and Technology, 54(14), 1313-1322. doi:10.1002/asi.10338Vaughan, L., & Yang, R. (2012). Web data as academic and business quality estimates: A comparison of three data sources. Journal of the American Society for Information Science and Technology, 63(10), 1960-1972. doi:10.1002/asi.22659Vaughan, L., & You, J. (2010). Word co-occurrences on Webpages as a measure of the relatedness of organizations: A new Webometrics concept. Journal of Informetrics, 4(4), 483-491. doi:10.1016/j.joi.2010.04.005Vaughan, L., Kipp, M. E. I., & Gao, Y. (2007). Why are Websites co-linked? The case of Canadian universities. Scientometrics, 72(1), 81-92. doi:10.1007/s11192-007-1707-y(The) Washington Post(2009), “It's official: Yahoo-Microsoft announce ten-year search/ad pact”, The Washington Post, available at: www.washingtonpost.com/wp-dyn/content/article/2009/07/29/AR2009072901108.html (accessed 27 February 2013).Wilkinson, D., Harries, G., Thelwall, M., & Price, L. (2003). Motivations for academic web site interlinking: evidence for the Web as a novel source of information on informal scholarly communication. Journal of Information Science, 29(1), 49-56. doi:10.1177/016555150302900105Zhang, Y. (2006). The Effect of Open Access on Citation Impact: A Comparison Study Based on Web Citation Analysis. Libri, 56(3). doi:10.1515/libr.2006.14

Espacio universitario español en la Web (2010): estudio descriptivo de instituciones y productos académicos a través del análisis de subdominios y subdirectorios

A descriptive analysis of the Spanish university system on the Net during 2010 is presented, through the identification, collection and analysis of a sample of entities (and associated URLs), at both the level of universities and university units (classified into institutions and products). An analysis is made of the number of institutions with valid URLs suited for cybermetric analysis purposes and the type of URL syntax (subdomain or subdirectory). Likewise, multi-domain and redirection practices are detected. For units with an identifiable area of knowledge (faculties, schools, departments, and research groups, centers and institutes), a thematic analysis is also carried out. The results indicate that the Spanish academic space has a complex structure, with abundant redirection and multi-domain practices, with a predominance of subdirectories in institutions and subdomains in products, and where the natural sciences have -by number of associated entities and URLs - a major presence.Se presenta un análisis descriptivo del sistema universitario español en la Red durante 2010, mediante la identificación, recopilación y análisis de una muestra de entidades y URLs asociados, tanto a nivel de universidad como de unidades universitarias clasificadas en instituciones y productos. Se analiza la cantidad de instituciones con URL válida a efectos de análisis cibermétricos, el tipo de sintaxis de URL subdominio o subdirectorio, así como la detección de prácticas de multidominios y redireccionamientos. Para las unidades con área del conocimiento identificable facultades, escuelas, departamentos, y grupos, centros e institutos de investigación, se realiza igualmente un análisis temático. Los resultados indican que el espacio académico español tiene una estructura compleja, donde abundan las prácticas de redireccionamiento, multidominio, con un predominio de los subdirectorios en las instituciones y subdominios en los productos, y donde las ciencias naturales tienen, en número de entidades y URLs asociados, una presencia mayoritaria.Orduña Malea, E. (2013). Espacio universitario español en la Web (2010): estudio descriptivo de instituciones y productos académicos a través del análisis de subdominios y subdirectorios. Revista española de Documentación Científica. 36(3):1-21. doi:10.3989/redc.2013.3.958S121363Adecannby, J. (2011). Web link analysis of interrelationship between top ten African universities and world universities. Annals of Library and Information Studies, vol. 58, 128-138.Aguillo, I. F., Granadino, B., Ortega, J. L., & Prieto, J. A. (2006). Scientific research activity and communication measured with cybermetrics indicators. Journal of the American Society for Information Science and Technology, 57(10), 1296-1302. doi:10.1002/asi.20433Bar-Ilan, J. (2005). The use of web search engines in information science research. Annual Review of Information Science and Technology, 38(1), 231-288. doi:10.1002/aris.1440380106Boudourides, M. A.; Sigrist, B.; Alevizos, P. D. (1999). Webometrics and the self-organization of the European information society. Rome Meeting of the SOEIS Project.Heimeriks, G., Hörlesberger, M., & Van den Besselaar, P. (2003). Scientometrics, 58(2), 391-413. doi:10.1023/a:1026296812830Kousha, K.; Horri, A. (2004). The relationship between scholarly publishing and the counts of academic inlinks to Iranian university Web sites: Exploring academic link creation motivations. Journal of Information Management and Scientometrics, vol. 1 (2), 13-22.Li, X. (2005). National and international university departmental Web site interlinking: a webometric analysis. University of Wolverhampton: Wolverhampton (UK).Martínez-Torres, M. D. R., Palacios-Florencio, B., Toral-Marín, S. L., & Barrero-García, F. J. (2011). Aplicación de algoritmos genéticos a la identificación de la estructura de enlaces en portales web. Revista española de Documentación Científica, 34(2), 232-252. doi:10.3989/redc.2011.2.779Noruzi, A. (2006). Web presence and impact factors for Middle-Eastern Countries. Online, vol. 30 (2), 22-28.Orduña-Malea, E., Serrano-Cobos, J., Ontalba-Ruipérez, J. A., & Lloret-Romero, N. (2010). Presencia y visibilidad web de las universidades públicas españolas. Revista española de Documentación Científica, 33(2), 246-278. doi:10.3989/redc.2010.2.740Orduña-Malea, E., Serrano-Cobos, J., & Lloret-Romero, N. (2009). Las universidades públicas españolas en Google Scholar: presencia y evolución de su publicación académica web. El Profesional de la Informacion, 18(5), 493-500. doi:10.3145/epi.2009.sep.02Ortega, José L. (2007). Visualización de la Web universitaria Europea: análisis cuantitativo de enlaces a través de técnicas cibermétricas. Madrid: Tesis Doctoral. Universidad Carlos III de Madrid.Ortega, J.-L., & Aguillo, I. (2007). La web académica española en el contexto del Espacio Europeo de Educación Superior: estudio exploratorio. El Profesional de la Informacion, 16(5), 417-425. doi:10.3145/epi.2007.sep.03Ortega, J. L., & Aguillo, I. F. (2008). Visualization of the Nordic academic web: Link analysis using social network tools. Information Processing & Management, 44(4), 1624-1633. doi:10.1016/j.ipm.2007.09.010Ortega, J. L., & Aguillo, I. F. (2009). Análisis estructural de la web académica iberoamericana. Revista española de Documentación Científica, 32(3), 51-65. doi:10.3989/redc.2009.3.689Ortega, J. L., Aguillo, I., Cothey, V., & Scharnhorst, A. (2007). Maps of the academic web in the European Higher Education Area — an exploration of visual web indicators. Scientometrics, 74(2), 295-308. doi:10.1007/s11192-008-0218-9Park, H. W., & Thelwall, M. (2006). Web-science communication in the age of globalization. New Media & Society, 8(4), 629-650. doi:10.1177/1461444806065660Pinto-Molina, M.; Alonso-Berrocal, J. L.; Cordón-García, J. A.; Fernández-Marcial, V.; García- Figuerola, C.; García-Marco, J.; Gómez-Camarero, C.; Zazo, Á. F.; Doucet, A. V. (2004). Análisis cualitativo de la visibilidad de la investigación de las universidades espa-olas a través de sus páginas web. Revista Espa-ola de Documentación Científica, vol. 27 (3), 345-370.Qiu, J., Chen, J., & Wang, Z. (2004). An analysis of backlink counts and Web Impact Factorsfor Chinese university websites. Scientometrics, 60(3), 463-473. doi:10.1023/b:scie.0000034387.76981.83Smith, A., & Thelwall, M. (2002). Scientometrics, 54(3), 363-380. doi:10.1023/a:1016030415822Tang, R., & Thelwall, M. (2003). U.S. academic departmental Web-site interlinking in the United States Disciplinary differences. Library & Information Science Research, 25(4), 437-458. doi:10.1016/s0740-8188(03)00053-7Tang, R., & Thelwall, M. (2004). Patterns of national and international Web inlinks to US academic departments: An analysis of disciplinary variations. Scientometrics, 60(3), 475-485. doi:10.1023/b:scie.0000034388.70594.ccThelwall, M. (2002). A comparison of sources of links for academic Web impact factor calculations. Journal of Documentation, 58(1), 66-78. doi:10.1108/00220410210425412Thelwall, M. (2002). An initial exploration of the link relationship between UK university Web sites. Aslib Proceedings, 54(2), 118-126. doi:10.1108/00012530210435248Thelwall, M. (2002). A research and institutional size‐based model for national university Web site interlinking. Journal of Documentation, 58(6), 683-694. doi:10.1108/00220410210448219Thelwall, M. (2003). Web use and peer interconnectivity metrics for academic web sites. Journal of Information Science, 29(1), 1-10. doi:10.1177/016555150302900101Thelwall, M. (2011). A comparison of link and URL citation counting. Aslib Proceedings, 63(4), 419-425. doi:10.1108/00012531111148985Thelwall, M.; Aguillo, I. F. (2003). La salud de las web universitarias espa-olas. Revista Espa-ola de Documentación Científica, vol. 26 (3), 291-305.Thelwall, M., Binns, R., Harries, G., Page-Kennedy, T., Price, L., & Wilkinson, D. (2002). Scientometrics, 53(1), 95-111. doi:10.1023/a:1014836021080Thelwall, M., Tang, R., & Price, L. (2003). Scientometrics, 56(3), 417-432. doi:10.1023/a:1022387105904Thelwall, M., Vaughan, L., & Björneborn, L. (2006). Webometrics. Annual Review of Information Science and Technology, 39(1), 81-135. doi:10.1002/aris.1440390110Thomas, O., & Willett, P. (2000). Webometric analysis of departments of librarianship and information science. Journal of Information Science, 26(6), 421-428. doi:10.1177/01655515000260060

Identifying the Invisible Impact of Scholarly Publications: A Multi-Disciplinary Analysis Using Altmetrics

A thesis submitted in partial fulfilment of the requirements of the University of Wolverhampton for the degree of Doctor of Philosophy.The field of ‘altmetrics’ is concerned with alternative metrics for the impact of research publications using social web data. Empirical studies are needed, however, to assess the validity of altmetrics from different perspectives. This thesis partly fills this gap by exploring the suitability and reliability of two altmetrics resources: Mendeley, a social reference manager website, and Faculty of F1000 (F1000), a post- publishing peer review platform. This thesis explores the correlations between the new metrics and citations at the level of articles for several disciplines and investigates the contexts in which the new metrics can be useful for research evaluation across different fields. Low and medium correlations were found between Mendeley readership counts and citations for Social Sciences, Humanities, Medicine, Physics, Chemistry and Engineering articles from the Web of Science (WoS), suggesting that Mendeley data may reflect different aspects of research impact. A comparison between information flows based on Mendeley bookmarking data and cross-disciplinary citation analysis for social sciences and humanities disciplines revealed substantial similarities and some differences. This suggests that Mendeley readership data could be used to help identify knowledge transfer between scientific disciplines, especially for people that read but do not author articles, as well as providing evidence of impact at an earlier stage than is possible with citation counts. The majority of Mendeley readers for Clinical Medicine, Engineering and Technology, Social Science, Physics and Chemistry papers were PhD students and postdocs. The highest correlations between citations and Mendeley readership counts were for types of Mendeley users that often authored academic papers, suggesting that academics bookmark papers in Mendeley for reasons related to scientific publishing. In order to identify the extent to which Mendeley bookmarking counts reflect readership and to establish the motivations for bookmarking scientific papers in Mendeley, a large-scale survey found that 83% of Mendeley users read more than half of the papers in their personal libraries. The main reasons for bookmarking papers were citing in future publications, using in professional activities, citing in a thesis, and using in teaching and assignments. Thus, Mendeley bookmarking counts can potentially indicate the readership impact of research papers that have educational value for non-author users inside academia or the impact of research papers on practice for readers outside academia. This thesis also examines the relationship between article types (i.e., “New Finding”, “Confirmation”, “Clinical Trial”, “Technical Advance”, “Changes to Clinical Practice”, “Review”, “Refutation”, “Novel Drug Target”), citation counts and F1000 article factors (FFa). In seven out of nine cases, there were no significant differences between article types in terms of rankings based on citation counts and the F1000 Article Factor (FFa) scores. Nevertheless, citation counts and FFa scores were significantly different for articles tagged: “New finding” or “Changes to Clinical Practice”. This means that F1000 could be used in research evaluation exercises when the importance of practical findings needs to be recognised. Furthermore, since the majority of the studied articles were reviewed in their year of publication, F1000 could also be useful for quick evaluations