How consistent are UK university rankings of subjects and their shifts? An exploration using a robust ranking aggregation method

Abstract

Rankings are ubiquitous, also within higher education settings. Guides are produced and they contain rankings for each subject provider. University rankings influence student choice of where to study. Moreover, league table rankings may be used by institutions to inform policy. As such, it is important to ensure these rankings are reliable. In this research, we applied a novel robust ranking algorithmic (RRA) method, commonly used in biosciences, to determine university rankings and compared this against the rankings from well-used university league tables (Guardian / Complete university guide). Although there was some correspondence between the RRA method rankings and the university league tables, there were also inconsistencies. Indeed, the median rank shift was 18 ranks for the Guardian University Guide and 11 ranks for the Complete University Guide. These findings were similar regardless of whether the focus was all institutions or those in the Russell Group. Additionally, there was little correspondence in ranking changes across years between the RRA and either well-used league table. Based on this, we argue that it is important to consider the reliability of university league tables when they are used in decision making, either by students or university staff