Our ambition is to identify, through the presentations and the discussion, a number of key insights and necessary actions for development of more and deeper partnerships between Higher Education Institutions and Employers

Murphy, Mike

English

Arrow@TUDublin

Technological University Dublin ARROW@TU Dublin Conference papers Engineering: Education and Innovation 2010 The Measurement of Success In External Engagement Activities Mike Murphy Technological University Dublin, mike.murphy@tudublin.ie Follow this and additional works at: https://arrow.tudublin.ie/engineduccon  Part of the Higher Education Commons, and the Other Engineering Commons Recommended Citation Murphy, M. (2010). The measurement of success in external engagement activities. Reap Conference, Cork Institute of Technology, 23 September. This Conference Paper is brought to you for free and open access by the Engineering: Education and Innovation at ARROW@TU Dublin. It has been accepted for inclusion in Conference papers by an authorized administrator of ARROW@TU Dublin. For more information, please contact arrow.admin@tudublin.ie, aisling.coyne@tudublin.ie. This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 License The Measurement of Success in External Engagement Activities Invited Talk – CIT REAP Conference 23 September 2010 Farmleigh House, Dublin.  Background “Our ambition is to identify, through the presentations and the discussion, a number of key insights and necessary actions for development of more and deeper partnerships between Higher Education Institutions and Employers.  “The session on the Measurement of Success in External Engagement Activities could, for example, focus on the metrics used in the examination of partnership [between] Higher Education Institutions and Employers.  It could, for example, examine incentive structures related to external engagement.  It could discuss the cultural and structural issues involved in the measurement of external engagement.  It would be most useful if you could draw out a small number of insights and actions from your personal experience, and respond briefly to the keynote address.” From REAP invitation letter   Chair, Distinguished Guests, I am delighted to be here today to address this topic.  In my prepared remarks I would like to develop the following key points. 1. 3rd mission represents both vital and necessary activities within the university 2. 3rd mission activities are broader than simply engagement with employers 3. Measuring 3rd mission activities presents challenges  Within higher education today, the topic of the measurement of institutional performance continues to grow in importance.  My remarks will not argue the point of whether such assessments are intrinsically good, or of discernable benefit.  Rather, I will address the question as to whether the measure of institutional performance should be broadened to include university third mission activities; followed by what should be included within 3rd mission activities and then ideas for how these activities might be measured.    My innate biases as an engineer may also show in these comments, for which I am not sure whether I should apologise or not!  A good starting point for this debate is an examination of the role of the university within higher education.  Note that I will use the descriptor ‘university’ to generically refer to all higher education institutions since, within an Irish higher education system, the distinction between quote ‘university’ and ‘non-university’ is contrived.  What should be the aims of the university?  Or should I say modern university; or research-led university, or research-informed university, or practice-led university, or enterprise university, etc, etc.  There is and should be diversity in higher education today.  The ethos, vision and goals of many universities may be quite different and the universities will benefit from being different.  Thus definition and context are important for understanding.   If we were to begin today with John Henry Newman´s core ideas regarding the value of liberal education put forward in his famous 19th century book “The Idea of a University”, then Newman would argue that the university is a hybrid educational environment which serves to educate students for life by means of “collegiality”, “enlargement of mind” and “acquisition of a philosophical habit of critical thinking”.  To use 21st century educational jargon, Newman is thus concerned with the learning outcomes of skills and competencies and not primarily with the content of what students learn.  But where and how should students get these skills within the walls of the 21st century university? The 21st century world that we live in is quite different than that of Newman’s 19th century.  To take the perspective of Rosalind Williams, in speaking on engineering education, she observes: “what engineers are being asked to learn keeps expanding along with the scope and complexity of the hybrid world.  Engineering has evolved into an open-ended Profession of Everything in a world where technology shades into society, into art, and into management, with no strong institutions to define an overarching mission.”  (Williams, 2002, p. 70).  If our current university learning environment is serving to educate professionals (for example, engineers) to expertise in very narrow specialisms, with a set of narrowly defined skills and competencies for pre-established jobs, then perhaps it is time to re-examine that learning environment.  For the complex world that our students live in today, we must educate “active, rigorous and flexible individuals, rather than skilled workers for pre-established jobs”.  (Braslavsky, 2002).   It therefore seems that while other arguments can and have been made for universities engaging in third mission activities, the modern university choosing to educate students in the professions must question, purely on pedagogic grounds, whether first and second mission activities are sufficient.   University activities have traditionally been based on two principal missions: teaching – i.e., creating an educated population and research – creating new knowledge.  However, since 1990, when Paul Romer published a landmark article, titled “Endogenous Technological Change,” in the Journal of Political Economy, economists have collaborated in developing a new theory of growth that puts knowledge — and not the traditional measurements of land or capital or labour or natural resources — at the centre of our understanding of the wellspring of economic change and progress.  Consequently, the twenty-first century economic paradigm is shifting toward putting knowledge first.  For Ireland, Europe and the World, increasingly, this means connecting higher education systems more closely to economic development strategies and we have entered a new, globalised era of higher education.  Therefore, the aims of the university within such a higher education system that is predominantly funded through public means, must be in pursuit of the benefits of society and the economy.  This new knowledge economy is making the traditional academic first and second missions ever more important.    Key Point # 1:  3rd mission represents both vital and necessary activities within the university.  Today a number of well-known university ranking methodologies/indicators are published to predominantly examine and measure the performance of the traditional first and second academic missions.  Criticisms of these rankings are many and valid, but these rankings are clearly addressing an information need on the part of their readers: i.e., the desire to know how one university, by some measure or other, ranks against others.  The fundamental problem with these rankings is that they measure what they can measure, rather than what should be measured; and then an arbitrary ranking methodology is applied against the measured data.    On 7th September,  the 2010 Times Higher Education World University Rankings methodology reported on the headline weightings given to five broad performance categories and included a new 3rd mission category – ‘Industry income – innovation’ with one indicator: ‘Research income from industry (per academic staff).  This metric is given a 2.5% weighting within the overall university score.  David F. Shaffer and David J. Wright observed in a March 2010 report entitled ‘A New Paradigm for Economic Development. How Higher Education Institutions Are Working to Revitalize Their Regional and State Economies’ that ‘globalization may lead universities to forge alliances and even establish campuses overseas, but institutions of higher education are anchored in their communities, and increasingly in a knowledge economy can serve as anchors for community development.  University leaders, meanwhile, increasingly see that the community environment has a direct impact on the marketability of their institutions as places to study, work, and invest.’   Partially for this reason, there has been a move towards recognising and acknowledging all the other activities of the university.  These include the exploitation and dissemination of created knowledge and capabilities into non-academic environments.  Activities such as public outreach and dissemination, community engagement, continuing education, knowledge and technology exchange, and international cooperation are services that demonstrate the university’s interaction with society.  These are all 3rd mission activities.    Key Point # 2:  3rd mission activities are broader than simply engagement with employers  How do we measure the impact of the university engaging in such 3rd mission activities?  As an engineer, I have been trained and educated to quantify and to measure.  Indeed, a pertinent quote comes from the Irishman William Thomson - Lord Kelvin, who once observed that "If you cannot measure it, you cannot improve it."   But let us spend a moment to examine this assertion.  The problem with measurements used for university rankings is that such measurement is quantitative.  Often the qualitative is ignored since, by its nature, it is hard to quantify.  Yet the qualitative responses are those that most often suggest how things can best be improved.  Paraphrasing Einstein, "Not everything that counts can be counted, and not everything that can be counted counts."  For us in DIT the biggest challenges with developing our Students Learning with Communities programme are not about the numbers of modules and those participating, which keep increasing, but about the quality of the experiences for all those involved, which is much harder to quantify.  A key factor in 3rd mission activities is the quality of the relationships which the university builds over time with community partners (e.g. a rich relationship can lead to the development of multi-annual, multi-disciplinary learning and research projects – like EPICS).  Yet this is one of the hardest things to quantify – but still should be measurable!  We thus need to uncouple measurement from numbers.   Key Point # 3:  Measuring 3rd mission activities presents challenges    • Teaching - the learning environment (30%) • Citations - research influence (32.5%) • Research - volume, income and reputation (30%) • International mix - staff and students (5%) • Industry income - innovation (2.5%).    The European Commission Leonardo Lifelong Learning Project on European indicators and ranking methodology for university third mission activities (E3M) is addressing the question of what 3rd mission activities should be measured and how they can be.  DIT is a partner in this Leonardo project.  The specific goals of this E3M project are as follows: (1)  To create European standard indicators to measure the effectiveness of third mission provision.  (2)  To validate the standard indicators.   (3)  To create a ranking methodology to benchmark European Third Mission Services of universities.   (4) To create good practices for institutional dialogue in a European Area of higher education framework.   The project proposes standardised indicators for the following dimensions of 3rd mission activities: Continuing Education, Technology Transfer & Innovation and Social Engagement.    For example, one of the continuing education processes: ‘Analysis of the Demand and Curriculum Design, can include the following indicators: • Total number of CE programmes active in that year (for implementation) • Number of CE programmes delivered which have a major award under European Higher Education system • Number of partnership CE programmes delivered in that year • Number of international CE programmes delivered in that year • Number of  funded CE training projects delivered in that year • Total number of the ECTS credits of the delivered CE programmes  The challenge with measurements is to broaden them sufficiently to enable and encourage diversity and excellence within university missions, yet not so broad as to provide little insight and value to any stakeholders.  One mechanism to encourage and improve university contributions to society would be to develop standard indicators combined with a ranking methodology, which will allow higher education institutions to improve the services they offer to society, including employers.  Such measures should provide incentives for universities to engage with the wider community.  This, in turn, should lead to the concept of community-led, or continuing education-led or 3rd mission-led university.    I would like to close with a quote from the US scholar Ed Zlotkowski, “Over the last fifteen years, much has been written about the need to rethink the role higher education can and should play in building a diverse democracy – a democracy whose graduates are not only capable of participating successfully in a knowledge-based economy but also of assuming their responsibilities as citizens in an increasingly interconnected world.  … Engagement points beyond student academic activities to describe the degree to which all academic activities – on the level of individual students and instructors, modules and programs of study, even entire institutions – have succeeded in creating educationally and socially productive partnerships with community-based organizations, especially organizations that address needs not met by private sector interests.”    

The Measurement of Success In External Engagement Activities

https://arrow.tudublin.ie/cgi/viewcontent.cgi?article=1041&context=engineduccon

The Measurement of Success In External Engagement Activities

Abstract

Similar works

Full text

Available Versions

Arrow@TUDublin