In the decades since science and technology measures were crafted and adopted by governments, R&D has increasingly taken place across national boundaries. This leaves a gap for policy makers in how to account for the benefits to national governments of supporting international collaboration in science. This article seeks to address this gap by suggesting a measure for the impact of international collaboration in science using fractionalized field-weighted citations and analysing these in relationship to public spending and researcher mobility

Baas, J.

Jonkers, K.

Wagner, C.S.

Whetsell, T.

English

Leiden University Scholary Publications

STI 2018 Conference Proceedings Proceedings of the 23rd International Conference on Science and Technology Indicators All papers published in this conference proceedings have been peer reviewed through a peer review process administered by the proceedings Editors. Reviews were conducted by expert referees to the professional and scientific standards expected of a conference proceedings. Chair of the Conference Paul Wouters Scientific Editors Rodrigo Costas  Thomas Franssen Alfredo Yegros-Yegros Layout Andrea Reyes Elizondo Suze van der Luijt-Jansen The articles of this collection can be accessed at https://hdl.handle.net/1887/64521  ISBN: 978-90-9031204-0 © of the text: the authors © 2018 Centre for Science and Technology Studies (CWTS), Leiden University, The Netherlands This ARTICLE is licensed under a Creative Commons Atribution-NonCommercial-NonDetivates 4.0 International Licensed Measure of National Return in International Science Cooperation Caroline S. Wagner*, Travis Whetsell**, Koen Jonkers ***, , Jeroen Baas*****wagner.911@osu.eduBattelle Center for Science, Engineering, and Public Policy, The Ohio State University, 1810College Road N., Columbus, OH 43210 (USA)** koen.jonkers@ec.europa.euKnowledge for Growth, Finance & Innovation Unit, Joint Research Centre, Brussels(Belgium)***Travis.Whetsell@gmail.comFlorida International University, Miami, FL (USA)****j.baas@elsevier.comElsevier, B.V., Amsterdam (Netherlands)Introduction Since the 1960s, governmental and intergovernmental organizations have published science and technology indicators for national evaluation (Godin, 2002). These works focus on national characteristics and do not fully account for the fact that international R&D increasingly takes place across national boundaries. Current metrics allow counts to be made of internationally coauthored papers, and citations to be attached to articles, but these measures do not disentangle the national contributions to the international effort, nor is it possible to assess the value to nations of participating in global collaboration. This paper explores the possibility of developing an indicator to better evaluate the benefits to nations of participating in international collaboration. The approach involves using fractionalized, field-weighted citation counts. We proceed by analyzing the correlation of this measure to national R&D investment, international collaboration and researcher mobility. Doing this in the form of a time series analysis allows us to go beyond Wagner et al (2018) and explore the potentially causal links between these factors. International collaboration in science has grown rapidly in the past three decades. Ganzi et al. (2012), Adams (2013), and Wagner et al. (2015) demonstrated the rapidly increasing numbers of internationally coauthored articles. When fractional counts are used to attribute contributions to participating countries, about 25% of all papers from OECD countries are nowadays internationally coauthored. Glänzel and Schubert (2001) and Glänzel and deLange (2002) showed that international collaboration is more highly cited. Cimini et al. (2016) showed that countries with high levels of international collaboration have a higher impact. These analyses suggest that international collaboration has a strong impact on science as a whole. However, the metrics leave a gap for those who wish to account for the benefits to nations of participating in growing amounts of international collaboration. This conference paper seeks to address this gap by proposing a measure of national benefit using fractionalized field-weighted citations, and analysing these measures in relationship to public spending and researcher mobility. 1380STI Conference 2018 · Leiden Data  The data gathered for this project, and the data sources, are shown in Table 1. Counts of all articles were collected for each country for each year from Scopus. With the total number of articles per year, we calculated the percentage of all articles that are internationally coauthored. The analysis uses fractional counts of the number of articles for specified years, as well as the percentage of all articles that are internationally coauthored. A fractional allocation of 3-year citation counts were calculated for each country for each year from Scopus data. In addition to the bibliometric data, we sought data on researcher mobility to contribute to assessing a country’s openness to international engagement, as well as spending on research and development (R&D) from OECD and GDP measures from the World Bank.  Table 1 - Data used in analysis, with source Data   Source Fractional count of number of internationally coauthored articles, 2007-2015*  FracPubs Scopus Fractional allocation of 3-year citation count for internationally coauthored articles, 2007-2015** FracFWCI Scopus Percent of all National Publications that have International Coauthorship Percent Intern Scopus Share of researchers who changed a national affiliation for a foreign affiliation in this year.  Outflows Scopus Share of researchers which changed a foreign affiliation to an affiliation in a specific nation.  Inflows Scopus Principal component index of Percent Intern, Outflows, Inflows Openness Calculated by team on the basis of above Scopus data Government Budget Allocation for R&D for 36 countries, 2007-2015 GBARD OECD Gross Domestic Product GDP World Bank  Bibliographic Information   The core data collection began with numbers of articles, numbers of international articles, and fractional counts of numbers of international articles from Scopus. The core data included aggregated FWCI from Scopus, which refers to "the ratio of citations received relative to the expected world average for the subject field, publication type and publication year" (SciVal, 2017) calculated by field and then aggregated into a single number for each country, which became a variable in the analysis. In addition, for each country for each year (2007-2015), we gathered FracFWCI assigns counts to countries proportionally, dividing the citation counts according to the frequency with which a country appears in the authors’ addresses. (We use FracFWCI as a measure of impact.) From these data we calculated the percent of all articles that were internationally coauthored as one variable in the analysis.  Mobility We sought to apply measures of openness of a country to international engagement, measured in part by researcher mobility. We added measures about the international mobility of a nation’s research workforce to percent of all papers that are internationally coauthored. These mobility data are calculated based on the percent of mobile researchers by examining new inflows and recent outflows. The “inflows”, or immigrant scientists, refers to the share of the authors who started publishing with an affiliation containing the country name under study 1381STI Conference 2018 · Leiden while initially using a different country as their institutional address. “Outflows”, or emigrant scientists, refers to the share of researchers who started publishing with the country under study as their institutional address followed by publications indicating (an) other(s) country or countries. Work by Sugimoto et at. (2017) showed that mobile researchers have higher citation counts.  Openness With the data on collaboration (percent international) and mobility (outflows and inflows), we created an openness index. The index is used to combine the three variables, which we found to be highly correlated. As a result, we calculated a Principal Component (PCN) index of measures to create a single measure called “openness” to indicate the extent of international engagement. The results of the PCN are included as a variable in Table 4 in the Results section. Government R&D Spending The goal was to understand the benefits to nations of participating in international collaboration, so we originally sought to add data about government R&D spending rather than including all data, which includes higher education and business spending. In a recent article (Wagner et al., 2018), we used OECD data on Government Budget Allocations or Outlays on R&D (GBAORD) by country, limiting the analysis to 34 countries reporting these data. However, GBAORD are only available for OECD member countries. For this paper, we sought to reproduce the findings we showed for one year (2013) to more years. We added data on Gross Domestic Product (RD/GDP) for 2007-2015 to represent investment from the World Bank.  Preliminary Analysis & Results  Once all the data were collected and organized, we created a model to test the findings by Wagner & Jonkers (2017) and Wagner et al. (2018) that openness was significantly correlated to impact as measured by FracFWCI. First, we examined statistical relationships as a correlation matrix, then we conducted cross-sectional analysis of each year. Following this, we analysed the panel data using a two-way fixed effects model. In this section, we present the bivariate correlations between all the variables used in the analysis. All of the variables have been standardized with mean zero and standard deviation of one in order to facilitate comparison of effect sizes. Since the correlations do not account for dependency in within-country variability over time, the correlations are only to be viewed as a starting point for the analysis. The top row for each variable shows the correlation, and the bottom row shows the p-value. All of the variables show strong and significant positive correlations with FracFWCI. 1382STI Conference 2018 · Leiden Table 2 – Correlation Matrix Variable N Mean Std Dev 1 2 3 4 5 6 7 8  1 - FracFWCI 312 -3.21E-12 1 1                                   2 - FracPubs 312 2.24E-11 1 0.260 1            <.0001         3 - Percent Intern. 312 3.21E-12 1 0.607 -0.267 1           <.0001 <.0001        4 - Outflows 312 1.92E-11 1 0.485 -0.145 0.778 1          <.0001 0.011 <.0001       5 - Inflows 312 -2.24E-11 1 0.635 -0.126 0.830 0.829 1         <.0001 0.026 <.0001 <.0001      6 - Openness 312 0 1 0.616 -0.191 0.929 0.929 0.948 1        <.0001 0.001 <.0001 <.0001 <.0001     7 - GDP 312 -9.62E-12 1 0.133 0.977 -0.320 -0.187 -0.197 -0.250 1       0.019 <.0001 <.0001 0.001 0.001 <.0001    8 - GBARD 312 -1.92E-11 1 0.200 0.979 -0.270 -0.147 -0.157 -0.205 0.980 1      0.000 <.0001 <.0001 0.009 0.005 0.000 <.0001   9 -GBARD/GDP 312 9.61E-12 1 0.608 0.295 0.263 0.107 0.126 0.176 0.234 0.317          <.0001 <.0001 <.0001 0.058 0.027 0.002 <.0001 <.0001                  Next, as discussed above, we created an “openness” measure since the percentage of publications that are international, as well as researcher inflows and outflows appear to be strongly correlated. They are theoretically conceptualized within “openness” for this project. We combined the three measures into a principal component. Table 3 shows the eigenvector loadings for the three measures on openness.  The component has an eigenvalue of 2.63. Table 3 – Principal Component Loadings for Openness Measures Table 4 shows the results of the regression models for each year between 2007 and 2015. Each model has FracFWCI as the dependent variable, and the model includes the fractional number of publications for each country, the openness principal component, and GBARD. Again all variables are standardised to facilitate comparison of effect sizes. The table shows that both the openness component and GBARD/GDP have strong significant positive relationships with FracFWCI across the entire time period. In 2007 GBARD/GDP has a stronger estimate than Openness, but this flips in 2009, and Openness continues to gain strength relative to GBARD across the period, supporting the hypothesis.   Intern. Perc 0.573512 Inflows 0.57317 Outflows 0.585286  1383STI Conference 2018 · Leiden Table 4 – Cross Sectional Regressions, 2007-2015 2007 2008 2009 2010 2011 2012 2013 2014 2015 Intercept 0 0 0 0 0 0 0 0 0 (0.114) (0.09) (0.102) (0.091) (0.103) (0.093) (0.097) (0.089) (0.084) FracPubs 0.074 0.141 0.195 0.206 0.252* 0.275** 0.324** 0.296** 0.311** (0.133) (0.104) (0.119) (0.102) (0.113) (0.101) (0.103) (0.095) (0.09) Openness 0.375** 0.522*** 0.535*** 0.579*** 0.555*** 0.633*** 0.69*** 0.668*** 0.74*** (0.119) (0.097) (0.109) (0.1) (0.11) (0.1) (0.103) (0.095) (0.09) GBARD/GDP 0.674*** 0.559*** 0.484*** 0.458*** 0.431*** 0.374*** 0.301** 0.382*** 0.314** (0.129) (0.104) (0.118) (0.104) (0.113) (0.102) (0.103) (0.095) (0.09) R2 0.614 0.74 0.679501 0.737274 0.664 0.717 0.71 0.748 0.774 N 33 35 34 35 35 36 34 34 35 P-value<0.05 = *, <0.01=**, <0.001***Next, Table 5 shows the results of the two-way fixed effects analysis. This model focuses on the within-country variability over time, in contrast to Table 4 which focused on cross-sectional between country variability. Table 5 shows an unexpected result, as the estimate on Openness has reversed sign, to become negative, apparently refuting the hypothesis that increases in openness in countries results in higher impact.  Table 5 – Two Way Fixed Effects Model, 2007-2015 FracFWCI Intercept -0.5(1.136)FracPubs 0.266(0.212)Openness -0.454***(0.077)GBARD/GDP 0.119** (0.036) Country (36) Fixed Year (9) Fixed R2 0.976 N 312 P-value<0.05 = *, <0.01=**, <0.001***1384STI Conference 2018 · Leiden The unexpected results in Table 5 prompted further inquiry into the comparison of between-country and within-country association of Openness and FracFWCI. Figure 1 shows the cross sectional scatterplot that pools all  years of data for all countries. The scatterplot shows the correlation between openness and impact, where the observations for each country are labelled with a specific symbol identified in the legend.  Figure 1 – Scatterplot, Openness and FracFWCI, Pooled 2007-2015   Figure 1 shows a positive slope for openness and FracFWCI. However, there is a good deal of within-country variability, illustrated with the country color coding, which indicates that, for many countries, the relationship between openness and FracFWCI has a negative slope. Figure 2 and Figure 3 show the USA and The Netherlands as examples of a country with a negative slope, refuting a positive relationship between openness and FracFWCI.   1385STI Conference 2018 · Leiden Figure 2 – Scatterplot for USA, Openness and FracFWCI Figure 3 – Scatterplot for Netherlands, Openness and FracFWCI Table 6 compares the number of countries having positive slopes with the number of countries having negative slopes. The number of countries with negative slopes outnumber those with positive slopes in the sample of countries.  1386STI Conference 2018 · Leiden Table 6 – Countries with positive and negative slopes for Openness and FracFWCI Positive/Flat Negative SWE USA SVN TUR NOR SVK MEX RUS KOR POL ITA NZL IRL NLD HUN LVA GRC LUX GBR JPN FIN ISR EST ISL ESP FRA DNK DEU BEL CZE AUS CHL ARG CHE CAN AUT Table 7 shows results for two-way fixed effects models for countries with (17 countries) positive and those countries with negative slopes (19 countries) over the nine-year period. The first model shows that Openness maintains a positive significant effect on FracFWI, but GBARD/GDP is no longer significant. The second models shows results consistent with the full model in Table 5, where there is a negative estimate for openness on impact and a positive estimate for GBARD on impact.  Table 7 – Two Way Fixed Effects, 2007-2015 Positive(FWCI) Negative(FWCI)Intercept 0.969*** -1.941(0.076) (1.596)FracPubs 0.553** 0.462(0.19) (0.296)Openness 0.19* -0.781***(0.073) (0.115)GBARD/GDP -0.036 0.264***(0.026) (0.056)Country Fixed FixedYear Fixed FixedR2 0.992 0.975 N 150 162 P-value<0.05 = *, <0.01=**, <0.001***1387STI Conference 2018 · Leiden Discussion This paper extends analysis from earlier work (Wagner and Jonkers, 2017 and Wagner et al., 2018) which introduced an index of openness of nations as a measure of engagement to test the possibility of measuring national benefit from international coauthorship and mobility. The earlier work had a number of limitations, particularly that the data were limited to a single year and consisted of cross-sectional correlations. Data on mobility and government spending were limited to a small set of developed nations, which is still the case in this conference paper. To improve upon the earlier work, this paper employs data over a nine-year period to provide a more robust test of the relationship between openness and impact. The idea of this additional test is to see if results would be robust across years; this included cross sectional analysis of each year as well as analysis of the decadal data. The goal of the work has been to assign proportional shares of output and impact to nations, to link that to spending, and to use the analysis to gain insight into the impact of international cooperation and mobility on national science. The cross-sectional analysis continues to show the relationship between openness and impact for years from 2008 – 2015. This means that on average, countries with higher openness also tend to have higher impact. However, the panel data do not support the hypothesis that increasing openness results in higher impact. We observe a mixed result, where some of the most advanced countries reveal a negative relationship between openness and impact. Other nations show a positive relationship.  Further analysis and discussion will consider possible explanations for these findings. The next steps in this analysis is to explore: 1) what type of countries do and what kind of countries do not benefit from increasing openness and 2) adding more countries to the analysis.  The authors welcome all comments.  Acknowledgement Thanks to Vincent Lariviere for comments on data analysis. 1388STI Conference 2018 · Leiden References Adams, J., 2013, Collaborations: The fourth age of research, Nature 497, 557–560 Gazni, A., Sugimoto, C. R., & Didegah, F. (2012). Mapping world scientific collaboration: Authors, institutions, and countries. Journal of the American Society for Information Science and Technology, 63(2), 323-335 Glänzel, W. & de Lange, C. 2002, A distributional approach to multinationality measures of international scientific collaboration, Scientometrics 54: 75.  Glänzel, W., & Schubert, A. (2001). Double effort= double impact? A critical view at international co-authorship in chemistry. Scientometrics, 50(2), 199-214. Godin, B. (2002). Outline for a history of science measurement. Science, technology, & human values, 27(1), 3-27. OECD 1963-2015, The Frascati Manual, Guidelines for collecting and reporting data on research and experimental development, OECD Publishing, Paris http://www.oecd.org/sti/inno/frascati-manual.htm OECD, 2015, OECD Science, Technology and Industry Scoreboard 2015 - Innovation for Growth and Society, http://www.keepeek.com/Digital-Asset-Management/oecd/science-and-technology/oecd-science-technology-and-industry-scoreboard-2015_sti_scoreboard-2015-en#.WexujUibvUM Sugimoto, C. R., Robinson-Garcia, N., Murray, D. S., Yegros-Yegros, A., Costas, R., & Larivière, V. (2017). Scientists have most impact when they're free to move. Nature, 550(7674), 29. Wagner, CS., Jonkers, K., 2017, Open countries have strong science, Nature 550, 7674 Wagner, C. S., Park, H. W., & Leydesdorff, L. (2015). The continuing growth of global cooperation Wagner, C. S., Whetsell, T., Baas, J., & Jonkers, K. (2018). Openness and impact of leading scientific countries. Frontiers in Research Metrics and Analytics, 3, 10. 1389

Measure of National Return in International Science Cooperation

https://scholarlypublications.universiteitleiden.nl/access/item%3A2729836/download

Measure of National Return in International Science Cooperation

Abstract

Similar works

Full text

Available Versions

Leiden University Scholary Publications