Ratings and rankings: Voodoo or Science?

Composite indicators aggregate a set of variables using weights which are understood to reflect the variables' importance in the index. In this paper we propose to measure the importance of a given variable within existing composite indicators via Karl Pearson's `correlation ratio'; we call this measure `main effect'. Because socio-economic variables are heteroskedastic and correlated, (relative) nominal weights are hardly ever found to match (relative) main effects; we propose to summarize their discrepancy with a divergence measure. We further discuss to what extent the mapping from nominal weights to main effects can be inverted. This analysis is applied to five composite indicators, including the Human Development Index and two popular league tables of university performance. It is found that in many cases the declared importance of single indicators and their main effect are very different, and that the data correlation structure often prevents developers from obtaining the stated importance, even when modifying the nominal weights in the set of nonnegative numbers with unit sum.Comment: 28 pages, 7 figure

The Leiden Ranking 2011/2012: Data collection, indicators, and interpretation

The Leiden Ranking 2011/2012 is a ranking of universities based on bibliometric indicators of publication output, citation impact, and scientific collaboration. The ranking includes 500 major universities from 41 different countries. This paper provides an extensive discussion of the Leiden Ranking 2011/2012. The ranking is compared with other global university rankings, in particular the Academic Ranking of World Universities (commonly known as the Shanghai Ranking) and the Times Higher Education World University Rankings. Also, a detailed description is offered of the data collection methodology of the Leiden Ranking 2011/2012 and of the indicators used in the ranking. Various innovations in the Leiden Ranking 2011/2012 are presented. These innovations include (1) an indicator based on counting a university's highly cited publications, (2) indicators based on fractional rather than full counting of collaborative publications, (3) the possibility of excluding non-English language publications, and (4) the use of stability intervals. Finally, some comments are made on the interpretation of the ranking, and a number of limitations of the ranking are pointed out

A longitudinal analysis of university rankings

Pressured by globalization and the increasing demand for public organisations to be accountable, efficient and transparent, university rankings have become an important tool for assessing the quality of higher education institutions. It is therefore important to carefully assess exactly what these rankings measure. In this paper, the three major global university rankings, The Academic Ranking of World Universities, The Times Higher Education and the Quacquarelli Symonds World University Rankings, are studied. After a description of the ranking methodologies, it is shown that university rankings are stable over time but that there is variation between the three rankings. Furthermore, using Principal Component Analysis and Exploratory Factor Analysis, we show that the variables used to construct the rankings primarily measure two underlying factors: a universities reputation and its research performance. By correlating these factors and plotting regional aggregates of universities on the two factors, differences between the rankings are made visible. Last, we elaborate how the results from these analysis can be viewed in light of often voiced critiques of the ranking process. This indicates that the variables used by the rankings might not capture the concepts they claim to measure. Doing so the study provides evidence of the ambiguous nature of university ranking's quantification of university performance.Comment: 26 page

A Comparative Analysis between Global University Rankings and Environmental Sustainability of Universities

[EN] Global University Rankings (GURs) intend to measure the performance of universities worldwide. Other rankings have recently appeared that evaluate the creation of environmental policies in universities, e.g., the Universitas Indonesia (UI) GreenMetric. This work aims to analyze the interaction between the Top 500 of such rankings by considering the geographical location of universities and their typologies. A descriptive analysis and a statistical logistical regression analysis were carried out. The former demonstrated that European and North American universities predominated the Top 500 of GURs, while Asian universities did so in the Top 500 of the UI GreenMetric ranking, followed by European universities. Older universities predominated the Top 500 of GURs, while younger ones did so in the Top 500 of the UI GreenMetric ranking. The second analysis demonstrated that although Latin American universities were barely present in the Top 500 of GURs, the probability of them appearing in the Top 500 of the UI GreenMetric ranking was 5-fold. We conclude that a low association exists between universities' academic performance and their commitment to the natural environment in the heart of their institutions. It would be advisable for GURs to include environmental indicators to promote sustainability at universities and to contribute to climate change.Muñoz-Suárez, M.; Guadalajara Olmeda, MN.; Osca Lluch, JM. (2020). A Comparative Analysis between Global University Rankings and Environmental Sustainability of Universities. Sustainability. 12(14):1-19. https://doi.org/10.3390/su12145759S1191214Dill, D. D., & Soo, M. (2005). Academic quality, league tables, and public policy: A cross-national analysis of university ranking systems. Higher Education, 49(4), 495-533. doi:10.1007/s10734-004-1746-8Shehatta, I., & Mahmood, K. (2016). Correlation among top 100 universities in the major six global rankings: policy implications. Scientometrics, 109(2), 1231-1254. doi:10.1007/s11192-016-2065-4Basu, A., Malhotra, D., Seth, T., & Kumar Muhuri, P. (2019). Global Distribution of Google Scholar Citations: A Size-independent Institution-based Analysis. Journal of Scientometric Research, 8(2), 72-78. doi:10.5530/jscires.8.2.12Mussard, M., & James, A. P. (2018). Engineering the Global University Rankings: Gold Standards, Limitations and Implications. IEEE Access, 6, 6765-6776. doi:10.1109/access.2017.2789326Olcay, G. A., & Bulu, M. (2017). Is measuring the knowledge creation of universities possible?: A review of university rankings. Technological Forecasting and Social Change, 123, 153-160. doi:10.1016/j.techfore.2016.03.029Moed, H. F. (2016). A critical comparative analysis of five world university rankings. Scientometrics, 110(2), 967-990. doi:10.1007/s11192-016-2212-yKivinen, O., Hedman, J., & Artukka, K. (2017). Scientific publishing and global university rankings. How well are top publishing universities recognized? Scientometrics, 112(1), 679-695. doi:10.1007/s11192-017-2403-1Alcaide, M. Á., De La Poza, E., & Guadalajara, N. (2019). Assessing the Sustainability of High-Value Brands in the IT Sector. Sustainability, 11(6), 1598. doi:10.3390/su11061598Massaro, M., Dumay, J., Garlatti, A., & Dal Mas, F. (2018). Practitioners’ views on intellectual capital and sustainability. Journal of Intellectual Capital, 19(2), 367-386. doi:10.1108/jic-02-2017-0033De Filippo, D., Sandoval-Hamón, L. A., Casani, F., & Sanz-Casado, E. (2019). Spanish Universities’ Sustainability Performance and Sustainability-Related R&D+I. Sustainability, 11(20), 5570. doi:10.3390/su11205570Trencher, G., Nagao, M., Chen, C., Ichiki, K., Sadayoshi, T., Kinai, M., … Yarime, M. (2017). Implementing Sustainability Co-Creation between Universities and Society: A Typology-Based Understanding. Sustainability, 9(4), 594. doi:10.3390/su9040594Sonetti, G., Lombardi, P., & Chelleri, L. (2016). True Green and Sustainable University Campuses? Toward a Clusters Approach. Sustainability, 8(1), 83. doi:10.3390/su8010083Zou, Y., Zhao, W., Mason, R., & Li, M. (2015). Comparing Sustainable Universities between the United States and China: Cases of Indiana University and Tsinghua University. Sustainability, 7(9), 11799-11817. doi:10.3390/su70911799An, Y., Davey, H., & Harun, H. (2017). Sustainability Reporting at a New Zealand Public University: A Longitudinal Analysis. Sustainability, 9(9), 1529. doi:10.3390/su9091529Blasco, N., Brusca, I., & Labrador, M. (2019). Assessing Sustainability and Its Performance Implications: An Empirical Analysis in Spanish Public Universities. Sustainability, 11(19), 5302. doi:10.3390/su11195302Alshuwaikhat, H., Adenle, Y., & Saghir, B. (2016). Sustainability Assessment of Higher Education Institutions in Saudi Arabia. Sustainability, 8(8), 750. doi:10.3390/su8080750Xiong, W., & Mok, K. H. (2020). Sustainability Practices of Higher Education Institutions in Hong Kong: A Case Study of a Sustainable Campus Consortium. Sustainability, 12(2), 452. doi:10.3390/su12020452Leal Filho, W., Emblen-Perry, K., Molthan-Hill, P., Mifsud, M., Verhoef, L., Azeiteiro, U. M., … Price, E. (2019). Implementing Innovation on Environmental Sustainability at Universities Around the World. Sustainability, 11(14), 3807. doi:10.3390/su11143807Brusca, I., Labrador, M., & Larran, M. (2018). The challenge of sustainability and integrated reporting at universities: A case study. Journal of Cleaner Production, 188, 347-354. doi:10.1016/j.jclepro.2018.03.292Alonso-García, S., Aznar-Díaz, I., Cáceres-Reche, M.-P., Trujillo-Torres, J.-M., & Romero-Rodríguez, J.-M. (2019). Systematic Review of Good Teaching Practices with ICT in Spanish Higher Education. Trends and Challenges for Sustainability. Sustainability, 11(24), 7150. doi:10.3390/su11247150Von Hauff, M., & Nguyen, T. (2014). Universities as Potential Actors for Sustainable Development. Sustainability, 6(5), 3043-3063. doi:10.3390/su6053043Roos, N., & Guenther, E. (2020). Sustainability management control systems in higher education institutions from measurement to management. International Journal of Sustainability in Higher Education, 21(1), 144-160. doi:10.1108/ijshe-01-2019-0030Caeiro, S., Sandoval Hamón, L. A., Martins, R., & Bayas Aldaz, C. E. (2020). Sustainability Assessment and Benchmarking in Higher Education Institutions—A Critical Reflection. Sustainability, 12(2), 543. doi:10.3390/su12020543Lehmann, M., Christensen, P., Thrane, M., & Jørgensen, T. H. (2009). University engagement and regional sustainability initiatives: some Danish experiences. Journal of Cleaner Production, 17(12), 1067-1074. doi:10.1016/j.jclepro.2009.03.013Salvioni, D. M., Franzoni, S., & Cassano, R. (2017). Sustainability in the Higher Education System: An Opportunity to Improve Quality and Image. Sustainability, 9(6), 914. doi:10.3390/su9060914Li, X., Ni, G., & Dewancker, B. (2019). Improving the attractiveness and accessibility of campus green space for developing a sustainable university environment. Environmental Science and Pollution Research, 26(32), 33399-33415. doi:10.1007/s11356-019-06319-zSuwartha, N., & Berawi, M. A. (2019). The Role of UI GreenMetric as a Global Sustainable Rankings for Higher Education Institutions. International Journal of Technology, 10(5), 862. doi:10.14716/ijtech.v10i5.3670Puertas, R., & Marti, L. (2019). Sustainability in Universities: DEA-GreenMetric. Sustainability, 11(14), 3766. doi:10.3390/su11143766Academic Ranking of World Universities-ARWUhttp://www.shanghairanking.com/ARWU-Methodology-2017.htmlQS Top University Rankingshttps://www.topuniversities.com/qs-world-university-rankings/methodologyTHE World University Rankingshttps://www.timeshighereducation.com/world-university-rankingsRanking Web de Universidades-Webometricshttp://www.webometrics.info/en/About_UsLiu, Z., Moshi, G. J., & Awuor, C. M. (2019). Sustainability and Indicators of Newly Formed World-Class Universities (NFWCUs) between 2010 and 2018: Empirical Analysis from the Rankings of ARWU, QSWUR and THEWUR. Sustainability, 11(10), 2745. doi:10.3390/su11102745Marginson, S. (2013). University Rankings and Social Science. European Journal of Education, 49(1), 45-59. doi:10.1111/ejed.12061Hauptman Komotar, M. (2019). Global university rankings and their impact on the internationalisation of higher education. European Journal of Education, 54(2), 299-310. doi:10.1111/ejed.12332Peters, M. A. (2017). Global university rankings: Metrics, performance, governance. Educational Philosophy and Theory, 51(1), 5-13. doi:10.1080/00131857.2017.1381472Hosier, M., & Hoolash, B. K. A. (2017). The effect of methodological variations on university rankings and associated decision-making and policy. Studies in Higher Education, 44(1), 201-214. doi:10.1080/03075079.2017.1356282Safón, V. (2019). Inter-ranking reputational effects: an analysis of the Academic Ranking of World Universities (ARWU) and the Times Higher Education World University Rankings (THE) reputational relationship. Scientometrics, 121(2), 897-915. doi:10.1007/s11192-019-03214-9Tuesta, E. F., Garcia-Zorita, C., Ayllon, R. R., & Sanz-Casado, E. (2019). Does a Country/Region’s Economic Status Affect Its Universities’ Presence in International Rankings? Journal of Data and Information Science, 4(2), 56-78. doi:10.2478/jdis-2019-0009Dobrota, M., & Dobrota, M. (2015). ARWU ranking uncertainty and sensitivity: What if the award factor was Excluded? Journal of the Association for Information Science and Technology, 67(2), 480-482. doi:10.1002/asi.23527Dowsett, L. (2020). Global university rankings and strategic planning: a case study of Australian institutional performance. Journal of Higher Education Policy and Management, 42(4), 478-494. doi:10.1080/1360080x.2019.1701853Rehman, M. A., Kashif, M., & Mingione, M. (2017). Corporate Social Responsibility and Sustainability (CSRS) Initiatives among European and Asian Business Schools: A Web-based Content Analysis. Global Business Review, 20(5), 1231-1247. doi:10.1177/0972150917737435Doğan, G., & Al, U. (2019). Is it possible to rank universities using fewer indicators? A study on five international university rankings. Aslib Journal of Information Management, 71(1), 18-37. doi:10.1108/ajim-05-2018-0118Siniksaran, E., & Satman, M. H. (2019). WURS: a simulation software for university rankings—software review. Scientometrics, 122(1), 701-717. doi:10.1007/s11192-019-03269-8Çakır, M. P., Acartürk, C., Alaşehir, O., & Çilingir, C. (2015). A comparative analysis of global and national university ranking systems. Scientometrics, 103(3), 813-848. doi:10.1007/s11192-015-1586-6Docampo, D., & Cram, L. (2016). Academic performance and institutional resources: a cross-country analysis of research universities. Scientometrics, 110(2), 739-764. doi:10.1007/s11192-016-2189-6Jöns, H., & Hoyler, M. (2013). Global geographies of higher education: The perspective of world university rankings. Geoforum, 46, 45-59. doi:10.1016/j.geoforum.2012.12.014UI GreenMetric World University Rankinghttp://greenmetric.ui.ac.id/Suwartha, N., & Sari, R. F. (2013). Evaluating UI GreenMetric as a tool to support green universities development: assessment of the year 2011 ranking. Journal of Cleaner Production, 61, 46-53. doi:10.1016/j.jclepro.2013.02.034Lauder, A., Sari, R. F., Suwartha, N., & Tjahjono, G. (2015). Critical review of a global campus sustainability ranking: GreenMetric. Journal of Cleaner Production, 108, 852-863. doi:10.1016/j.jclepro.2015.02.080Ragazzi, M., & Ghidini, F. (2017). Environmental sustainability of universities: critical analysis of a green ranking. Energy Procedia, 119, 111-120. doi:10.1016/j.egypro.2017.07.054Marrone, P., Orsini, F., Asdrubali, F., & Guattari, C. (2018). Environmental performance of universities: Proposal for implementing campus urban morphology as an evaluation parameter in Green Metric. Sustainable Cities and Society, 42, 226-239. doi:10.1016/j.scs.2018.07.012Drahein, A. D., De Lima, E. P., & Da Costa, S. E. G. (2019). Sustainability assessment of the service operations at seven higher education institutions in Brazil. Journal of Cleaner Production, 212, 527-536. doi:10.1016/j.jclepro.2018.11.293Parvez, N., & Agrawal, A. (2019). Assessment of sustainable development in technical higher education institutes of India. Journal of Cleaner Production, 214, 975-994. doi:10.1016/j.jclepro.2018.12.305Undetermined Scalehttps://www.google.es/maps/@39.4657727,-0.8023025,3zQGIS Geographic Information Systemhttps://qgis.orgGao, X. (Andy), & Zheng, Y. (2018). ‘Heavy mountains’ for Chinese humanities and social science academics in the quest for world-class universities. Compare: A Journal of Comparative and International Education, 50(4), 554-572. doi:10.1080/03057925.2018.1538770Zhou, Y., & Wu, J. (2016). The Game Plan: Four Contradictions in the Development of World Class Universities from the Global South. TED EĞİTİM VE BİLİM, 41(184). doi:10.15390/eb.2016.6152Alba-Hidalgo, D., Benayas del Álamo, J., & Gutiérrez-Pérez, J. (2018). Towards a Definition of Environmental Sustainability Evaluation in Higher Education. Higher Education Policy, 31(4), 447-470. doi:10.1057/s41307-018-0106-

Hierarchical Cluster Analysis: A New Type of Ranking Criteria Based on ARWU Ranking Data

The advent of big data leads to many applications of Machine Learning techniques. University rankings is one of the applicable domains, which is currently playing a crucial role in the assessment of the universities\u27 performance. Currently, the rankings are usually carried out by some authoritative ranking institutions by means of weighting techniques and the results are conveyed in numerical rankings. Three of the most famous university ranking institutions have been introduced from a technical perspective. However, these institutions have been proven to be subjective in relation to their data selection and weighting method

Ranking institutions within a university based on their scientific performance: A percentile-based approach

Over the recent years, the subject of university rankings has attracted a significant amount of attention and sparked a scientific debate. However, few studies on this topic focus on elaborating the scientific performance of universities’ institutions, such as institutes, schools, and faculties. For this reason, the aim of this study is to design an appropriate framework for evaluating and ranking institutions within a university. The devised methodology ranks institutions based on the number of published papers, mean normalized citation score (MNCS), and four percentile-based indicators using the I-distance method. We applied the proposed framework and scrutinized the University of Belgrade (UB) as the biggest and the best-ranked university in Serbia. Thus, 31 faculties and 11 institutes were compared. Namely, an in-depth percentile-based analysis of the UB papers indexed in the Science Citation Index Expanded (SCIe) and the Social Science Citation Index (SSCI) for the period 2008-2011 is provided. The results clearly show considerable discrepancies in two occasions: first, when it comes to the question of leading author, and second, when it comes to analyzing the percentile rank classes (PRs) of groups of faculties

Shedding Light on the Doing Business Index: a Machine Learning Approach

Background: The World Bank (WB) acknowledged the importance of business regulatory environment and therefore created a metric which ranks 190 countries based on their level of business regulation for domestic firms measured by the Doing Business Index (DBI). Objectives: The question which attracted our attention is whether all the observed entities should be given the same weighting scheme. Methods/Approach: The approach we propose as an answer is two-fold. First, we cluster the countries covered by the DBI. In the next step, we apply the statistical multivariate Composite I-distance Indicator (CIDI) methodology to determine new, data-driven weights for each of the retained clusters. Results: The obtained results show that there is a difference between the weighting schemes proposed by the CIDI methodology. Conclusions: One can argue that one weighting scheme does not fit all the observed countries, meaning that additional analyses on the DBI are suggested to explore its stability and its weighting scheme

Search Engine Optimization

This Special Issue book focuses on the theory and practice of search engine optimization (SEO). It is intended for anyone who publishes content online and it includes five peer-reviewed papers from various researchers. More specifically, the book includes theoretical and case study contributions which review and synthesize important aspects, including, but not limited to, the following themes: theory of SEO, different types of SEO, SEO criteria evaluation, search engine algorithms, social media and SEO, and SEO applications in various industries, as well as SEO on media websites. The book aims to give a better understanding of the importance of SEO in the current state of the Internet and online information search. Even though SEO is widely used by marketing practitioners, there is a relatively small amount of academic research that systematically attempts to capture this phenomenon and its impact across different industries. Thus, this collection of studies offers useful insights, as well as a valuable resource that intends to open the door for future SEO-related research