A Suggested Method for the Measurement of World-Leading Research (Illustrated with Data on Economics)

Countries often spend billions on university research. There is growing interest in how to assess whether that money is well spent. Is there an objective way to assess the quality of a nation's world-leading science? I attempt to suggest a method, and illustrate it with modern data on economics. Of 450 genuinely world-leading journal articles, the UK produced 10%, and the rest of Europe slightly more. Interestingly, more than a quarter of these elite UK articles came from outside the best-known university departments. The proposed methodology could be applied to almost any academic discipline or nation.Research Excellence Framework (REF), peer-review, United Kingdom, European economics, evaluation, science, citations, Research Assessment Exercise (RAE)

Asian top universities in six world university ranking systems

There are a variety of ranking systems for universities throughout the different continents of the world. The majority of the world ranking systems have paid special attention toward evaluation of universities and higher education institutions at the national and international level. This paper tries to study the similarities and status of top Asian universities in the list of top 200 universities by these world ranking systems. Findings show that there are some parallelisms among these international rankings. For example it was found some correlations between QS-Webometrics rankings (R= 0.78); QS-THE rankings (R= 0.53); and Shanghai-HEEACT rankings (R= 0.58). The highest correlation rate belongs to QS-Webometrics (R=0.78). The findings show no evidence to prove that the origin country of ranking system has any bias toward the rank of universities of its own country among other countries. For instance QS ranking of the United States classifies many universities of China and Japan as top Asian universities. HEEACT Ranking System of Taiwan includes just one university of Taiwan in the high ranking category (as other rankings do). Shanghai Ranking of China assigns a lower grade to universities of China and Hong Kong in comparison with QS ranking of the USA. Finally, some suggestions are made to improve the benefits of the ranking systems in order to promote the situation of higher education in the world, and recommendations for combining the indicators of these ranking systems to have a more comprehensive one for the world

How should peer-review panels behave?

Many governments wish to assess the quality of their universities. A prominent example is the UK’s new Research Excellence Framework (REF) 2014. In the REF, peer-review panels will be provided with information on publications and citations. This paper suggests a way in which panels could choose the weights to attach to these two indicators. The analysis draws in an intuitive way on the concept of Bayesian updating (where citations gradually reveal information about the initially imperfectly-observed importance of the research). Our study should not be interpreted as the argument that only mechanistic measures ought to be used in a REF

Please don't aim for a highly cited paper

Citation-based metrics are important in determining careers, so it is unsurprising that recent publications advise prospective authors on how to write highly cited papers. While such publications offer excellent advice on structuring and presenting manuscripts, there are significant downsides, including: restrictions in the topics researched, incentives to misconduct and possible detriments to motivation, innovation and collegiality. Guides to writing highly cited papers also assume that all citations are equal, ignoring new directions in bibliometric research identifying ‘quality’ and perfunctory citations. Rather than pursuing citations, with the uncertainty about their significance and the potential negative consequences, authors may fare better by following evidence from several disciplines indicating that persistence, a focused research program, good methodology and publishing in relevant journals are more important in career development and disciplinary influence than the odd star paper. Research administrators could encourage such steps by considering innovative new multivariate assessments of research productivity, including assessing social impact

A suggested method for the measurement of world-leading research (illustrated with data on economics)

Countries often spend billions on university research. There is growing interest in how to assess whether that money is well spent. Is there an objective way to assess the quality of a nation's world-leading science? I suggest a method, and illustrate it with data on economics. Of 450 genuinely world-leading journal articles, the UK produced 10% and the rest of Europe slightly more. Interestingly, more than a quarter of these UK articles came from outside the best-known university departments. The proposed methodology could be applied to almost any academic discipline or nation

Peer Review Practices

Peer Review (PRev) is among the oldest certification practices in science and was designed to prevent poor research from taking place. There is overall agreement that PRev is the most solid method for the evaluation of scientific quality. Since PRev spans the boundaries of several societal communities, science and policy, research and practice, academia and bureaucracy, public and private, the purposes and meaning of this process may be understood differently across the communities. In Europe, internationally competitive research activities take place in large superstructures as well as in small, insufficiently funded university departments; research can be publicly or privately funded; the purpose may be applied research often with a focus on the needs of regional industry, or purely ‘blue-sky’ research. In current report we focused mainly in on PRev of grant applications, the analysis has been carried out on the basis of PRev related literature analysis (Thomson Reuters, Union Library Catalogues, Google Scholar, and reports of selected research funding organisations)