"In FEAST’s previous Discussion Papers we have demonstrated a series of bibliometric methodologies useful for policy-makers to quickly attain a grasp of their country’s international research positioning and relative performance. This paper extends these concepts by applying several
measures at once in order to demonstrate more clearly, from a broad country-level perspective, those countries with which Australia gains a comparative advantage by
collaborating. This technique allows for policy-makers to target only those countries that provide a clear justification for doing so, in particular by combining quality measures with output volumes." - page 2Australian National Universit

Faletič, Rado

The Australian National University

Published by the Forum for European-Australian Science and Technology cooperation         This project is supported by International Science Linkages established under the Australian Government’s innovation statement, Backing Australia’s Ability, and has received funding from the European Commission’s Seventh Framework Programme, Capacities: International Cooperation (222747).  info@feast.org +61 2 6125 7885 www.feast.org Beryl Rawson building 13, The Australian National University, Canberra ACT 0200 Australia Reconciling Quality and Quantity FEAST Discussion Paper 4/11 25 February 2011 Dr Rado Faletič1                                                     1 Dr Rado Faletič, Executive Director, FEAST, director@feast.org, +61 2 6125 7885. Reconciling Quality and Quantity  25 February 2011 Forum for European-Australian Science and Technology cooperation 2/8 Introduction In FEAST’s previous Discussion Papers we have demonstrated a series of bibliometric methodologies useful for policy-makers to quickly attain a grasp of their country’s international research positioning and relative performance. This paper extends these concepts by applying several measures at once in order to demonstrate more clearly, from a broad country-level perspective, those countries with which Australia gains a comparative advantage by collaborating. This technique allows for policy-makers to target only those countries that provide a clear justification for doing so, in particular by combining quality measures with output volumes. Methodology The cooperation pay-off matrix developed by Matthews et al [1] allows for two countries to be compared using any given capability index, see Figure 1. This framework was used in Matthews et al [2], using the Relative Citation Impact (RCI)2 as the capability index, to demonstrate Australia’s standing relative to EU Member States and other selected nations. That paper implemented a simple ‘traffic light’ system to clearly indicate which countries, in which fields, stand to gain a pull-up opportunity via collaboration with Australia. However, the data, from an Australian policy-maker’s perspective, fails to consider issues such as the volume of research in each field being conducted by each nation, and hence the appropriateness of these results for the                                                    2 Relative Citation Impact, or RCI, is the average citations per paper for a given country in a given field divided by the global average number of citations per paper in that field. purpose of developing Australian Government policies to target specific nations. Below we detail a metric that considers both the RCI and the Proportion of Global Output (P), a measure, expressed as a percentage, of a country’s total number of publications in a field compared with the global output. Using the same data as published in Matthews et al [1],3 we introduce a new measure, the Relative Significance (RS), defined by combining RCI and P as follows: RS =RCI! + P 4 !2  This is a standard form of measurement of two variables – RCI and P may be considered the horizontal and vertical axes of a Cartesian plot, hence this equation is simply an implementation of Pythagoras theorem. The inclusion of the numerical values of 2 and 4 act to normalise RS such that when RCI is equal to 1 and P is equal to 4%, RS is then unity. The value of 4% was used as the threshold for P as Australia’s output in its best performing fields, in terms of RCI, constitute approximately 4% of the global total. The Relative Significance measure reconciles the relative quality as well as quantity of publications, and will enable policy-makers to better discern between countries of similar RCI but vastly different volume of output. As such, RS greater than 1 provides a compelling case for collaboration, as it consists of one of these three possibilities: 1. RCI≳1and P≳4%, 2. RCI≫1 and P<4%, 3. RCI<1 and P≫4%.                                                    3 The data set is Thomson-Reuters’ National Science Indicators (NSI) for the years 2003-2007, acquired by the Australian Government Department of Innovation, Industry, Science and Research (DIISR). This data set was filtered to consider only countries that produced at least 100 papers over the sample period in each given field.  Country Y Capability Index > 1.0 Country Y Capability Index < 1.0 Country X Capability Index > 1.0 X: Forge-ahead opportunity Y: Forge-ahead opportunity X: Pull-down risk Y: Pull-up opportunity Country X Capability Index < 1.0 X: Pull-up opportunity Y: Pull-down risk X: Catch-up opportunity Y: Catch-up opportunity Figure 1: Bilateral cooperation pay-off matrix. Reconciling Quality and Quantity  25 February 2011 Forum for European-Australian Science and Technology cooperation 3/8 All three possibilities present the potential to either be pulled-up in terms of citation impact, or in terms of access to larger national research and innovation systems, or both. Findings For the purpose of illustration, three fields were selected for analysis based on three key positions in the pay-off matrix as described in Matthews et al [1]. These were: Agricultural Sciences, Geosciences, and Materials Science, and their significance in the pay-off matrix are shown in Figure 2. Figures depicting the complete set of results are contained in the Appendix. Figure 3 depicts a typical spread of excellence, that is, there is a high degree of variation between countries and between fields, clearly dominated by countries with more highly developed economies. Note the absence of the massively populated China and India. Compare this with the percentage of global publications in Figure 5, which clearly shows where the bulk of publications are being produced, the USA, along with the other ‘major’ contributors to global output. China and India clearly appear on this graph.    EU-27 strength RCI > 1.1 EU-27 borderline RCI 0.9-1.1 EU-27 weakness RCI < 0.9 Australia strengths RCI > 1.1 Geosciences   Australian borderline RCI 0.9-1.1 Agricultural Sciences Materials Science  Australia weakness RCI < 1.1    Figure 2: Three chosen fields, and their positions in the Australia/EU-27 pay-off matrix. Figure 3: Relative Citation Impact (RCI) for Agricultural Sciences, Geosciences and Materials Science where RCI ≥  1. 1.00 1.10 1.20 1.30 1.40 1.50 1.60 Australia Austria Belgium Bulgaria Canada China Cyprus Czech Republic Denmark Estonia EU-27 Finland France Germany Greece Hungary India Ireland Italy Latvia Lithuania Luxembourg Malta Netherlands New Zealand Poland Portugal Romania Slovakia Slovenia South Africa Spain Sweden USA UK Agricultural Sciences Geosciences Materials Science Reconciling Quality and Quantity  25 February 2011 Forum for European-Australian Science and Technology cooperation 4/8   Figure 5: Proportion of Global Output (P) for Agricultural Sciences, Geosciences and Materials Science where P ≥  4%. 0% 5% 10% 15% 20% 25% 30% 35% 40% Australia Austria Belgium Bulgaria Canada China Cyprus Czech Republic Denmark Estonia EU-27 Finland France Germany Greece Hungary India Ireland Italy Latvia Lithuania Luxembourg Malta Netherlands New Zealand Poland Portugal Romania Slovakia Slovenia South Africa Spain Sweden USA UK Agricultural Sciences Geosciences Materials Science 1.00 2.00 3.00 4.00 5.00 6.00 7.00 Australia Austria Belgium Bulgaria Canada China Cyprus Czech Republic Denmark Estonia EU-27 Finland France Germany Greece Hungary India Ireland Italy Latvia Lithuania Luxembourg Malta Netherlands New Zealand Poland Portugal Romania Slovakia Slovenia South Africa Spain Sweden USA UK Agricultural Sciences Geosciences Materials Science Figure 4: Relative Significance (RS) for Agricultural Sciences, Geosciences and Materials Science where RS ≥  1. Reconciling Quality and Quantity  25 February 2011 Forum for European-Australian Science and Technology cooperation 5/8 Now looking at the plot of RS for the same fields, in Figure 5, we clearly see features of both RCI and P coming through. That is, the ‘major’ science powers still dominate the landscape (USA, UK, Germany, France), but note how a number of ‘smaller’ nations begin to appear in selected fields, eg Belgium, Denmark, Finland, the Netherlands and Sweden, due to the strong quality of their publications. Conversely, despite both China and India appearing below average on the RCI scale their sheer volume of publications in several fields, notable Materials Science, gives them a RS above unity. Looking at Europe, from the full RS results in Figure 8 in the Appendix, Australia could best achieve pull-up opportunities from focusing bilateral attention primarily on the European science powerhouses of France, Germany and the UK, and secondly on Italy, the Netherlands and Denmark, and thirdly on Spain and Sweden. There might also be occasion to consider thematic targets with Austria (Physics), Belgium (Agricultural Sciences and Clinical Medicine), Finland (Agricultural Sciences and Physics), Hungary (Space Science), Latvia (Clinical Medicine) and Portugal (Space Science). Looking multilaterally the EU-27 still shows a compelling pull-up capability. Outside of Europe it is obvious that the USA is an attractive partner country, as are Canada and China. India, however, appears to only offer a pull-up opportunities in Chemistry and Materials Science. Conclusions By combining both quality and quantity indices into a single measure we have demonstrated a more useful “real world” measure than relying solely on either one. This technique might usefully be applied to other indices, particularly the volume of bilateral collaborations (measured via joint publications) between Australia and target countries. References [1] Matthews, M., Biglia, B., and Murphy, B. (2009). A Comparison of Australian and European Union research performance profiles. FEAST Discussion Paper 2/09. http://www.feast.org/index/document/2 [2] Matthews, M., Biglia, B., Glennie, K. and Harris, P. (2010). Mapping Australia’s research strengths from an international perspective. AUS-ACCESS4EU Discussion Paper. http://www.access4.eu/_img/article/D1_4_Australia_research_strength.pdf Certain data included herein are derived from the National Science Indicators® prepared by THOMSON REUTERS®, Inc. (Thomson®), Philadelphia, Pennsylvania, USA: © Copyright THOMSON REUTERS® 2006. All rights reserved. Reconciling Quality and Quantity  25 February 2011 Forum for European-Australian Science and Technology cooperation 6/8 Appendix The charts on the following pages summarise the complete results for RCI, P and RS. These charts are reproduced here primarily in order to demonstrate broad country perspectives, rather then specific country strengths. Note that the detail in each chart may be examined more closely on a computer screen by simply zooming in (300% ought to be suitable). Charts for specific countries and/or fields may be requested from FEAST via info@feast.org.  Figure 6: Relative Citation Impact (RCI) data for all countries where RCI ≥  1. This view clearly highlights the broad RCI strength found in both Denmark and the Netherlands. 1.00 1.10 1.20 1.30 1.40 1.50 1.60 1.70 Australia Austria Belgium Bulgaria Canada China Cyprus Czech Republic Denmark Estonia EU-27 Finland France Germany Greece Hungary India Ireland Italy Latvia Lithuania Luxembourg Malta Netherlands New Zealand Poland Portugal Romania Slovakia Slovenia South Africa Spain Sweden USA UK Agricultural Sciences Biology & Biochemistry Chemistry Clinical Medicine Computer Science Ecology/Environment Economics & Business Education Engineering Geosciences Immunology Law Materials Science Mathematics Microbiology Molecular Biology & Genetics Multdisciplinary Neurosciences & Behavior Pharmacology Physics Plant & Animal Science Psychology/Psychiatry Social Sciences, general Space Science Reconciling Quality and Quantity  25 February 2011 Forum for European-Australian Science and Technology cooperation 7/8  Figure 7: Proportion of Global Output (P) for all countries where P ≥  4%. Note that for a number of fields the USA and the EU-27 produce more than half of the total global output. 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Australia Austria Belgium Bulgaria Canada China Cyprus Czech Republic Denmark Estonia EU-27 Finland France Germany Greece Hungary India Ireland Italy Latvia Lithuania Luxembourg Malta Netherlands New Zealand Poland Portugal Romania Slovakia Slovenia South Africa Spain Sweden USA UK Agricultural Sciences Biology & Biochemistry Chemistry Clinical Medicine Computer Science Ecology/Environment Economics & Business Education Engineering Geosciences Immunology Law Materials Science Mathematics Microbiology Molecular Biology & Genetics Multdisciplinary Neurosciences & Behavior Pharmacology Physics Plant & Animal Science Psychology/Psychiatry Social Sciences, general Space Science Reconciling Quality and Quantity  25 February 2011 Forum for European-Australian Science and Technology cooperation 8/8  Figure 8: Relative Significance (RS) for all countries where RS ≥  1.  1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 Australia Austria Belgium Bulgaria Canada China Cyprus Czech Republic Denmark Estonia EU-27 Finland France Germany Greece Hungary India Ireland Italy Latvia Lithuania Luxembourg Malta Netherlands New Zealand Poland Portugal Romania Slovakia Slovenia South Africa Spain Sweden USA UK Agricultural Sciences Biology & Biochemistry Chemistry Clinical Medicine Computer Science Ecology/Environment Economics & Business Education Engineering Geosciences Immunology Law Materials Science Mathematics Microbiology Molecular Biology & Genetics Multdisciplinary Neurosciences & Behavior Pharmacology Physics Plant & Animal Science Psychology/Psychiatry Social Sciences, general Space Science 

Reconciling quality and quantity

Abstract

Similar works

Full text

Available Versions

The Australian National University