Maclean’s University Rankings 2018: Misled Again

Abstract

Using a protocol of statistical tools and procedures, we provide an empirical examination of Maclean’s 2018 magazine rankings of Canadian universities based on the analysis of approximately 40 indices. The ranking system utilized by Maclean’s may cause a variety of very real consequences, not only ill-fated to student consumers’ well-being but also to the universities and their surrounding communities. Analyses were divided by Canadian university category (Medical Doctoral, Comprehensive, and Primarily Undergraduate), and included: (a) Spearman rho correlations of index scores to final ranks, (b) Wilcoxon rank-sum tests to compare higher- versus lower-ranked institutions, and (c) a cluster analysis to derive comparable families of similar institutions. Canadian universities, in reality, resemble and relate to each other in a way very different than the system of final rank ordering and formal classification that Maclean’s claims to use. Overall, the analysis showed (a) nonsignificant and largely inconsistent relationships based on the correlation between index scores and final ranks and (b) trivial differences between higher and lower-ranked institutions. Additionally, and consistent with analysis from years prior, we found that Maclean’s annual analyses using a rank-based approach to evaluate universities offers inadequate practical use, different from their continually advertised intentions of providing consumers all they need to know to choose the right university