“As a general observation, we should probably be alert to the different conditions and requirements that operate between e.g.'big science'(that might require heavy investment in equipment and facilities and large teams of researchers) and research e.g. in the humanities and social sciences that might have different features and requirements.”--

Bridges, David

English

Nazarbayev University Repository

78 QUALITY ASSURANCE IN EDUCATION & RESEARCH AND ACCOUNTABILITY David Bridges As a general observation, we should probably be alert to the different conditions and requirements that operate between e.g.'big science'(that might require heavy investment in equipment and facilities and large teams of researchers) and research e.g. in the humanities and social sciences that might have different features and requirements. 1. What makes a university or university department an attractive and supportive environment for academic researchers? What might provide evidence of an excellent research environment? How do you build or develop such an environment? Here are some candidates for inclusion in a response - some of them drawn from experience in the UK Research Assessment Exercise, which required panels to assess the quality of the research environment in individual university departments. Which of these (or other features) would you regard as most important? • Mentoring and support is offered to early career researchers • There are funds available to support attendance at research conferences etc. • There is a strong nucleus of research and especially doctoral students • The department has a strong track record of attracting external research funding for research • There are regular research seminars at which colleagues discuss each other's work • There are team and collaborative projects and publications • There are regular opportunities for sabbatical terms • Departmental leaders protect staff from over burdensome bureaucracy and meetings • Teaching loads are l imited to 50%/ 60%/ 7 0 % (?) of the tota l workload • There are regular visitors from other national and international HE institutions • The coffee room/ restaurant and etc. are a buzz of research conversation • There is an excellent library and readily available access to on-line resources • (For example in scientific fields) there are state of the art laboratories and technical facilities 2. What expectations should university or departmental leaders have of the evidence individual members of faculty should be able to offer of their research productivity and quality? Some universities express these expectations in terms of the number of publications eg in peer reviewed/ or ' international ' journals but what is a reasonable expectation? And are there not issues about responsibility to publish in journals in the local language accessible to one's own citizens? (See Bridges 2006) Some universities express this in terms of research grants won or applied for. (This applies especially but not exclusively to those in senior positions). In some universities (the University of Ghent in Belgium, for example) such expectations are enshrined in individual contracts and faculty are liable for dismissal if they do not achieve 79 what is set down. More often they get drawn into some sort of guidance and mentoring system, though the threat of dismissal or early retirement hangs over them. 3. Against what criteria do/should we use to assess the quality of these research products? The European Education Research Quality Indicators (EEROI) project (www.eerqi.eu ), supported by a range of national research associations and leading publishers, proposed the fol lowing criteria of quality for the assessment of research publications: Rigour i.e. it has to be conducted systematically in line wi th established or persuasively proposed research methods; argument and inference and interpretation need to be logically drawn etc. Originality i.e. it has to offer something new or innovative Significance i.e. it has to make a contribution of some importance to the field (eg policy, practice, innovative technology) to which it relates Style i.e. it is expressed clearly, articulately, intell igibly - perhaps even elegantly Integrity i.e. it is the authentic work of the author and acknowledges debts where appropriate; it is conducted ethically and where appropriate in line wi th established professional or research ethical codes. Are these the criteria that you would apply? 4. How do we make a valid and reliable assessment of this quality against such criteria? The UK Research Assessment Panels that assessed research quality across a l l subject areas and a l l higher education institutions in 2008 (see Bridges 2009a) decided that there was no adequate short cut to engaging directly wi th the research that was offered for assessment (in general four papers drawn from the previous seven years work by each submitted member of staff) i.e. by reading it. Many systems have sought to find a less demanding and more mechanical form of assessment by e.g. • Some simply count the number of papers published by individuals in particular kinds of journals (e.g.'peer reviewed' journals/international ' journals, journals that appear in the Web of Science). • Some countries (including Australia and France) have generated fierce controversy by trying to rate journals according to what some people have judged as their importance. (A European Community sponsored project aimed at creating a similar index for the humanities collapsed into rather satisfying ignominy!). • Some have tried to employ citation indices as a proxy for quality notwithstanding the arbitrary composition of such indices and the ambivalences in the messages they provide. • At one stage the EEROI project, assisted by some clever linguists from Xerox Grenoble tried to identify machine recognisable semantic features of high quality publications, but these attempts were defeated by among other things the fact that judgements about originality and significance are relational i.e. they require the assessor to look at one piece of writ ing against a l l sorts of understanding about its context and other sources that machine readers simply did not have and/ or could not apply. 80 My own argument (which there is not space to develop here but see Bridges, D. 2009b) is that none of these proxies provide a valid and reliable alternative to an expert and moderated peer review reading of the text). But this view is linked to the requirements of an assessment that becomes rather fine grained at the top end (and this may be material to the discussion) The UK RAE functioned not just to identify basically adequate research (an assessment for which acceptance by a reputable peer reviewed journal might serve as an adequate proxy) but to distinguish among such research what was excellent or outstanding by national standards and then what was world class. It was this level of discrimination for which the various proxies proposed seemed seriously inadequate: informed connoisseurship or judgement by practised peer reviewers seemed to be the only way. 5. System wide, do you achieve research quality by being selective i.e. by concentrating research funding on institutions/ departments judged to produce high quality research or by distributing such investment across the higher education sector? The 'fine grained'judgement that I refer to above served in the UK research assessment system (and was subsequently taken up in several other administrations , including e.g. Hong Kong) not so much for the purpose of assessing in any recognisable way individual researchers, but to assess - along wi th considerations of the quality of the research environment indicated above - - the quality of work of individual university departments. This assessment then served in turn to determine what funding that university would receive over the next five years or so to support its research. This approach was quite explicitly linked to a policy decision taken in the context of an expanding higher education system to concentrate research funding in institutions that could demonstrate research excellence: it was and remains a policy of'research selectivity'. The consequence is that a university whose departments do not perform we l l in the research assessment (the'Research Assessment Exercise'in 2008, the Research Excellence Framework in 2014) do not receive that proportion of funding that would otherwise support the share of faculty t ime dedicated to research. The consequence for individual university academics in these circumstances is that they have effectively to earn their entire salary through income derived from their teaching and that they have no allowance of t ime for research. These exercises are conducted every 5 to 7 years, and it is easy to see how, to some extent they become self-fulf i l l ing: the universities that start off doing we l l get the main share of funding for research and the next t ime round they tend to do we l l again. In practice the consequence in the UK is that something like 8 0 % or more of research funding for higher education goes to the golden triangle ' of Oxford - Cambridge — London. The question our forum might consider is the balance of benefit to the country and to higher education of such 'research selectivity'. Further reading More about the UK research assessment exercises can be found on the web-site of the Higher Education Funding Council at www.rae.ac.uk and www.ref.ac.uk. There is more information about the EEROI initiative at www.eerqi.eu. 8: Some of the points made in this piece are developed in the fol lowing works by the author: Bridges, D. (2006) 'The international and the excellent in educational research' in P.Smeyers & M Depaepe (2006) (eds.) Educational Research: why what works doesn't work, Dordrecht, Springer. Bridges, D. (2009a) 'Assessing the quality of research in higher education: the UK Research Assessment Exercise' in ed.T.Besley, Assessing the quality of research in higher education: a comparative study, Rotterdam, Sense Publications. Bridges, D. (2009b) Research quality assessment: impossible science, possible art? British Educational Research Journal, 35:4 pp. 497-517. 

QUALITY ASSURANCE IN EDUCATION & RESEARCH AND ACCOUNTABILITY

https://nur.nu.edu.kz/bitstream/123456789/300/1/QUALITY%20ASSURANCE%20David%20B.pdf

QUALITY ASSURANCE IN EDUCATION & RESEARCH AND ACCOUNTABILITY

Abstract

Similar works

Full text

Available Versions

Nazarbayev University Repository