Hazelkorn, Ellen

English

Arrow@TUDublin

Technological University Dublin ARROW@TU Dublin Other resources Centre for Social and Educational Research 2007-10-30 Learning to Live with League Tables and Rankings Ellen Hazelkorn Technological University Dublin, ellen.hazelkorn@tudublin.ie Follow this and additional works at: https://arrow.tudublin.ie/cseroth  Part of the Educational Assessment, Evaluation, and Research Commons, and the International and Comparative Education Commons Recommended Citation Hazelkorn, E.: Learning to live with league tables and rankings. Presentation given at the International Rankings Expert Group Meeting. Shanghai, 30 October, 2007. This Presentation is brought to you for free and open access by the Centre for Social and Educational Research at ARROW@TU Dublin. It has been accepted for inclusion in Other resources by an authorized administrator of ARROW@TU Dublin. For more information, please contact yvonne.desmond@tudublin.ie, arrow.admin@tudublin.ie, brian.widdis@tudublin.ie. This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 License Learning to Live with League Tables and RankingsEllen HazelkornDirector, and Dean of the Faculty of Applied ArtsDirector, Higher Education Policy Research Unit (HEPRU) Dublin Institute of TechnologyIREG3 ConferenceShanghai, October 2007Themes1. Overview of International Study 2. How Rankings Impact on HEIs and HE1. Institutional Perspectives 2. Other Evidence 3. What’s next? 1.Overview of International Study International Study Conducted in association with IMHE (OECD) and IAU –using their membership lists. Email questionnaires sent to leaders/senior administrators in June-September 2006. 639 questionnaires sent, with some unquantifiable ‘snowballing’ 202 replies received 31.6% response rate Respondent Profile (N=202) Age: 36% post 1970 24% 1945-1969 40% pre 1945 83% publicly funded Institutional type 30.4% teaching intensive 19.3% research informed 29.2% research intensiveGlobal Distribution41 countries, N=155 Australia Austria Belguim CanadaChina DenmarkFinland Germany  Greece IcelandIndia IranIraq IrelandItaly JapanJordan Keny aLebanon LithuaniaMex ico MongoliaNetherlands N Cy prusNorw ay ParaguayPhilippines PortugalRw anda Saudi ArabiaSerbia Slov eniaSouth Africa SpainSri Lanka Sw edenSw itzerland ThailandTurkey UKUSA2. How Rankings Impact on HEIs/HE?  Institutional Perspectives'The University itself is ranked among the top UK universities for the quality of its teaching' `Top of the … Student Satisfaction table' ‘Our position is clearly the second Finnish University in international rankings’‘The number one destination for international students studying in Australia’ ‘Institution accredited by FIMPES, Excelencia académica SEP, x Place in academic program of...’Playing the Rankings Game Despite methodological concerns strong perception that… Rankings help maintain/build institutional position and reputation Good students using rankings to ‘shortlist’, especially at postgraduate level Stakeholders using rankings to influence funding, sponsorship, and recruitment Benefits and advantages flow from high ranking HEIs taking results very seriously…Popularity and Purpose of Ranking Use of national rankings on the rise, but worldwide rankings have wider penetration. Over 70% respondents identified ‘providing comparative information’ as the primary purpose of LTRS However, there is a differentiation between the target audience and user of such surveys:  Target audience: students and public opinion User: public opinion, government, parents and industryRanking Status Significant gap between current and preferred rank  93% and 82%, respectively, want to improve their national or international ranking. 58% respondents not happy with current institutional ranking  Current ranking: 3% of all respondents are nationally ranked 1st in their country, but 12% want to be so ranked;  No respondents are internationally ranked 1st, but 3% want to be so ranked 70% of all respondents wish to be in top 10% nationally, and 71% want to be in top 25% internationally.Maintaining Position and Reputation Rankings play a critical role in enabling/facilitating HEIs to maintain and build institutional position and reputation.  While answers dependent upon ‘happiness with position’, almost 50% use their institutional position for publicity purposes: press releases, official presentations, website. 56% have a formal internal mechanism for reviewing their  56% by the Vice Chancellor, President or Rector 14% by the Governing AuthorityOver 40% of respondents said they considered an HEI’s rank prior to entering into discussions about:  international collaborations academic programmes research student exchanges 57% said they thought LTRS were influencing willingness of other HEIs to partner with them. 34% said LTRS were influencing willingness of other HEIs to support their institution’s members of academic/professional organisations. Peer-benchmarkingInfluence and Impact Strategic and Operational Decision-making Academic Programmes Research Activity  Student Recruitment Marketing Strategy  Partnership and Collaboration  Investment Stakeholder and Public OpinionInfluence on Key Stakeholders (1)CommentsBenefactors •‘It totally depends on the result of ranking’•‘Benefactors don’t want to be help or associated with losers. They want their image to be associated with winners only’•‘To-date, only universities [not all HEIs] have benefitted from benefactors’•‘They feel reassured supporting us’•‘Pride’Collaborators ‘We feel an improvement’‘Good for reputation’Current Faculty ‘Faculty feels honoured’‘Its easier to induce an improvement with department head whose rankings have been declining’Employers ‘Degree holders from universities with good reputation have better chances to get a job (and visa versa)’‘They feel reassured; those not open to us become more receptive’‘Employers can get confused if rankings do not reflect the real quality difference’Funding Agencies‘Assessment of institutional performance’Increase in quantity and size of funding to promote excellence in teaching and research’‘Have less pretext to deny funding’Influence on Key Stakeholders (2)CommentsFuture Faculty •‘Recruitment will be easier because of good reputation’•‘Make standards for appointment/promotion more clear and transparent’•‘Attractiveness’Government ‘Assessment of institutional performance’‘Local government is inclined to spend additional money for an excellent university’‘Less pretext for obstacles; more doors opened’Industry ‘Creates more research centres around the university’‘Investment decision’‘Better perception and disposition for collaboration’Parents ‘Benchmark for judging the best university’‘Advise their children to go to highly ranked university’‘Particularly in international markets where status and prestige are considered in decision-making’Partnerships ‘Establishment of international co-operation’‘Good for reputation’ at ‘international level’ Students ‘Benchmark for judging the best university’‘More students are willing to come to the campus’‘High profile students usually apply to high profile universities’‘…Students give too much weight to rankings without knowing the methodology’‘Influence at the margin (probably applies to the other categories as well)’‘Pride (actively shown in public forums)’ Actions Arising (1) 63% respondents have taken strategic, organisational, managerial or academic actions in response to the results Of those,  Overwhelming majority took either strategic or academic decisions and actions Only 8% respondents indicated they had taken no actionActions Arising (2)ExamplesStrategy •‘Indicators underlying rankings are explicit part of target agreements between rector and faculties’• ‘Have become part of a SWOT analysis•‘Organise benchmarking exercises’•‘Investment in weaker areas•‘Letting us know that we have to work more on our publicity and quality’Organisation • ‘A position in the controlling department of the administration has been established to deal with indicator improvements and ranking’• ‘Reorganisation of structure’• ‘Have organised investigation team’•‘Regular observation of rankings and methods; supervision of the data delivery to ranking projects; continuous observation of indicators of other universities’•‘Renewed emphasis on the accuracy/amount of data gathered and shared with 3rd parties’Management •‘Rector enforces the serious and precise processing of ranking as well as control of the relevant indicators’•‘Improvement of the results has become a target in the contract between presidency and departments’• ‘Development of better management (budgetary) tools for supporting fields of excellent research’Academic • ‘Deans and faculties are increasingly sensitized for ranking results and underlying indicators’•‘Results of rankings are regarded in the construction of the new study structure’•‘Improve teaching and learning’•‘Formulation of explicit demands for the productivity of the individual researcher’•‘Increase English language programmes’• ‘More scholarships and staff appointments’Other Actions ‘Driven us to consider unhelpful merger proposals’ ‘Made us spend money bolstering demand in key overseas markets to counter league tables’ ‘We have developed a set of internal research output indicators…we do internal benchmarking’ ‘Made us devote time to restoring our damaged feelings’Ideal ‘League Table’ Objective: Give fair and unbiased picture of the strengths/weaknesses Provide student choice for a programme and institution Enhance accountability and quality Metrics:  Teaching Quality, Staff/Student Ratio, Employment, Research, Publications, Research Income, PhDs, Finances, Student Life, Citations, Selectivity, Mission, and Library Using institutional or publicly available data or questionnaires  Institutional level  Undertaken by independent research organisation or accreditation agencyPerceptions of ImpactPositive impact if highly rated:  ‘Decent rankings may help raise/reassure awareness of institution/department/program and help support their activities’ ‘Foreign universities are interested in the fact that we are one of the three best private universities in our country’  ‘Installation of a privately funded department of real estate management’ by a benefactor/sponsor in response to rankings …but potentially harmful if reverse is true:  ‘Denial of collaboration because of a bad position in the Shanghai Ranking’  ‘Local newspapers write that local government should not spend more money for our university’ ‘Decline in enrolment’ Other Evidence ‘You should hold a degree from a Times top 100 university ranked at no 33 or higher’ ‘[Ireland] should aim to have two universities in the top 20 worldwide by 2013’Impact on Students & Recruitment Evidence is limited, but trends appearing High rankings  rise in applications (NY Times, 2007) Rank important for US high-ability students (Griffith/Rask, 2007) UK, Germany and New Zealand (Clarke, 2007; Federkeil, 2002) Ranking important for international recruitment/mobile postgraduate market (EAIE)Impact on Stakeholders US Governing Boards (Levin, 2002)  75% pay attention to US N&WR  68% Boards discuss rankings; 71% for half an hour or more. Employers favour graduates from more highly ranked HEIs (UK) (University of Sussex, 2006) State appropriations per student in public colleges are responsive to rankings (US) (Zhe Jin, 2007) Almost all universities chosen for Deutsche Telekom professorial chairs used rankings as evidence of research performance (Spiewak, 2005)US HEI Views 76% LTRS somewhat or very important  51% attempted to improve their rankings 50% used rankings as internal benchmarks  35% announced results in press releases or on the web 4%   established task force or committee to address rankings 20% ignored them (Levin, 2002, p14)Institutional (re)Action University administrators: ‘most engaged and obsessively implicated’ (Keller, 2007) Recruit students who will be ‘assets’ in terms of maintaining and enhancing rank (Clarke, 2007) HEIs making extensive investments to influence ‘student input’ metric (Brewer et al, 2002) 88% identified retention rates;  84% alumni-giving;  75% graduation rates;  71% entry scores;  63% faculty compensation;  31% student-faculty ratio.  25% improve educational expenditure 7% improve research capacity  (Levin, 2002)3. What’s Next?Conclusions Strong perception that benefits/advantages flow from high ranking Influence goes beyond ‘traditional’ student audience  Growing influence on public opinion, government and industry; Influence policymaking, e.g. classification of institutions, allocation of research funding, accreditation; HEIs taking results very seriously, and making changes Embedding league tables within strategic decision-making Making structural and organisational changes: Publicity and marketing. Institutions behaving rationally – becoming what is measured. Observations Rankings and League Tables gained popularity because they (appear to) fulfil particular purposes and needs Accordingly, ’concerns’ were easily ignored/shrugged off with reference to individual institution’s score or broader objectives (e.g. benchmarking, strategic planning) But, international experience replicated by/through literature and earlier US studyIncreasing evidence suggests wider usage, impact and influence   Implications for HEIs and Systems  Re-structuring HE systems, nationally and internationally Market Mechanisms and Global Competition Accountability and Transparency        Quality Assurance and Accreditation Institutional Benchmarking, Worldwide Ranking and League Tables Pace of HR reform likely to quicken as governments believe reform will lead to more competitive and better (more highly ranked) HEIs Ambiguous Tendencies Increasing vertical stratification w/ growing gap between elite and mass education Despite support for inter-institutional collaboration, in a competitive environment, ‘elite’ institutions may see little benefit working with/helping ‘lesser’ institutions.  Greater ‘mission’ convergence and undervaluing of institutional diversity  Growing Worldwide Differentiation Geo-political implications for developed and developing economies/societiesellen.hazelkorn@dit.ie

Learning to Live with League Tables and Rankings

https://arrow.tudublin.ie/cgi/viewcontent.cgi?article=1002&context=cseroth

Learning to Live with League Tables and Rankings

Abstract

Similar works

Full text

Available Versions

Arrow@TUDublin