Using citation data of 557 Nobel prize winning papers and the same number of their non-prize winning counterparts in the same journal issues, we examined if the prize-winning papers have higher academic disruption than their counterparts. The results show that overall, the former group is significantly more highly-cited but not more disruptive than the latter. Moreover, the results are not consistent with existing knowledge that the numbers of authors and references negatively correlate with the disruption of papers

Li, Jiang

Shi, Dongbo

Wei, Chunli

Zhao, Zhenyue

English

IDEALS @ Illinois

Nobel-Prize-winning papers are significantly more highly-cited but not more disruptive than non-prize-winning counterparts

Illinois Digital Environment for Access to Learning and Scholarship Repository

Nobel Prize Winning Papers are Significantly More Highly-Cited but not More Disruptive than Non-prize Winning Counterparts Chunli Wei1, Zhenyue Zhao1, Dongbo Shi2, and Jiang Li1 1 Nanjing University, Nanjing, Jiangsu 210023, China 2 Shanghai Jiao Tong University, Shanghai 200030, China lijiang@nju.edu.cn Abstract. Using citation data of 557 Nobel prize winning papers and the same number of their non-prize winning counterparts in the same journal issues, we examined if the prize-winning papers have higher academic disruption than their counterparts. The results show that overall, the former group is significantly more highly-cited but not more disruptive than the latter. Moreover, the results are not consistent with existing knowledge that the numbers of authors and references negatively correlate with the disruption of papers.  Keywords: Disruption, Scientific Papers, Nobel Prize, Citations. 1 Introduction Academic disruption refers to the degree to which each work disrupts the field of sci-ence or technology to which it belongs by introducing something new that eclipses at-tention to previous work upon which it has built [1,2]. According to Wu et al. [2] and its foundation [1], citations received by a paper were classified into three categories, i: citations given to the focal paper from papers which did not cite the references of the focal paper; j: citations given to the focal paper from papers which also cited the refer-ence(s) of the focal paper; and k: citations given to the reference(s) of the focal paper from papers which did not cite the focal paper. The disruption d of the focal paper is,                                                        (1) This measure is undefined for papers that received no citations. This calculation of dis-ruption, once being introduced, has attracted researchers’ attention in multiple fields who implemented it and probed historical literature to seek potential patterns and dig out implications for producing future disruptive studies [3-5]. Disruptive studies are a powerful means for broadening and exploring new research areas, which, in turn, dis-rupts existing academic linkages. With these disruptive studies, new research directions 2 and existing research streams could be converged together and further, enrich the entire academic network. Established following the will of Alfred Nobel, the Nobel Prize was embedded with the world’s highest prestige in the corresponding fields and has still been vigorous after the time-lapse of over a century. In recent years, the efforts of researchers such as Li et al. [6] and Chan, Önder & Torgler [7] have brought significant advancement in the availability of data for the Nobel laureates and their publications. This feasibility has enabled others to explore more traceable patterns governing the producing of elite sci-ence. Clarivate Analytics, for example, even developed a tool for predicting future No-bel laurates utilizing these datasets [8]. 2 Methodology We implemented the dataset of Nobel prize winning papers collected and shared by Li et al. [6], which included the records of 874 prize-winning papers for 545 Nobel laurates between 1900 and 2016. These papers were recognized from the references of the Nobel lecture and further checked under multiple criteria in order that those are the key pub-lications for the prize winning. In order to track their citation trajectories throughout the years after publication, we searched for these papers in the Web of Science (WoS) database using their DOIs (digital object identifier, a persistent identifier for journal articles, research reports, etc.) and eventually, 704 papers were found while the missing rest were excluded. Due to the missing of article details in the WoS, there are 147 prize winning papers for which we could not pair them up with non-prize winning counter-parts and they were thus eliminated. As a result, a control group of 557 non-Nobel prize winning papers was constructed by randomly selecting another paper in the same issue of the same journal for each of the 557 Nobel prize winning papers. Based on the cita-tion relationships, we collected ~0.8 million papers that cited the 1,114 papers and ~10 million papers that cited the references of the 1,114 papers, in the WoS. This study adopted equation 1 to quantify the academic disruption of scientific pa-pers. The disruption and citations for the prize-winning and non-prize winning papers were compared using paired samples t-test. 3 Results As shown in Figure 1, the result of the paired samples t-test also shows no evidence that the 557 prize winning papers are more disruptive, because the disruption of the prize winning group averages 0.236 while that of the control group averages 0.239 and two-tail paired samples t-test showed p=0.898. Next, we compared the numbers of au-thors of the two paper groups and identified that the mean authors of prize winning and non-prize winning papers are 4.662 and 2.404, respectively, and there is a significant difference between them with p=0.000 from paired samples t-test. We also compared the numbers of references of the two paper groups, and found that prize winning papers have more references than non-prize winning counterparts with mean values 15.964 and 12.371, respectively, as shown in Table 1. 3 Considering the lasting period time of the dataset and the growth of team size and references over time, we equally split 557 samples into two subgroups and their periods are hence 1900-1963 and 1964-2016. Results of t-test show that the prize winning pa-pers have significantly higher disruption (0.293 vs 0.155), more authors (2.262 vs 1.856) and more references (12.298 vs 7.989) than their non-prize winning counterparts during 1900-1963. Whereas for period 1964-2016, prize winning papers have lower disruption (0.178 vs 0.322) and more references (19.616 vs 16.738) than the non-prize-winning counterparts, and still have significant difference in the number of authors (7.053 vs 2.950). Figure 1. Distribution of the d values of prize winning and non-prize winning papers, the x-axis denotes the ranking of d values from the highest to lowest, and the y-axis indicates d values (left shows 557 pairs between 1900 and 2016, right top shows 278 pairs between 1900 and 1963 and the right bottom shows 279 pairs between 1964-2016) There exist significant differences between the two groups of 557 papers in the number of total citations. The prize winning papers received an average of 1344.115 citations while that of the control group papers is merely 61.047 (p<0.01, paired samples t-test), the former being over 20 times the number of the latter. Next, we scrutinized the dif-ference of i, j and k in both groups of papers. As shown in Table 1, there are significant differences of i, j and k between the two groups by paired samples t-test, and it is inter-esting that the values of I and j for the prize winning group are approximately 20 times their counterparts, but the value of k for the former group is two times latter group.  4    Table 1. Descriptive statistics of the citation measures Measure Mean  (prize winning group) Mean  (control group) Significance (paired samples t-test) disruption 0.236 0.239 0.898 authors 4.662 2.404 0.000 references 15.964 12.371 0.000  citations 1344.115 61.047 0.000 i 998.339 45.052 0.000 j 345.776 15.995 0.008 k 14285.220 7401.695 0.000  4 Summary By exploring the citations received by the 557 Nobel prize winning papers and their non-prize winning counterparts in the same issue of the same journal, we found that although the former group received approximately 20 times the citations of the latter group, there is no significant difference between the disruption values of the two groups. Furthermore, the results did not confirm existing findings by Wu et al. [2] that the numbers of authors and references negatively correlate with the disruption of the papers. Eventually, during 1900-1963 period, prize winning papers own more disrup-tion, while non-prize winning papers between 1964-2016 have significantly higher dis-ruption.  The above results initiated thinking about the newly proposed indicator ‘disruption’. It differentiated Nobel prize winning papers from ordinary publications in the WoS by categorizing them into the top 2% in terms of d values. However, it made no difference when we compared the prize winning with their non-prize winning counterparts in the same issues of the same journals, most of which are top journals. Moreover, our find-ings are different from Wu et al. (2019)’s when we considered the number of references and authors. For example, during 1900-1963, the 278 prize winning papers have more references and more authors than their non-prize winning counterparts and are expected to have lower disruption according to the negative correlation uncovered by Wu et al. [2], but the results are not in line with their claims.  One reason is whether Nobel prize winning papers are supposed to be ‘disruptive’. For example, Youyou Tu, the 2015 Nobel Prize laureate in Physiology or Medicine, discovered effective treatment of malaria utilizing artemisinin, which is regarded as a significant breakthrough in the field and a significant health improvement for people of tropical developing countries. Perhaps research disruption is only one of the reasons for the Nobel Prize winning, multiple factors will be considered by the committee, such as the benefits of Tu’s research towards human-beings.  5 Acknowledgement. The authors acknowledge the National Natural Science Founda-tion of China (NSFC Grant No. 71673242) for financial support.  References 1. Funk, R. J., Owen-Smith, J.: A dynamic network measure of technological change. Man-agement Science, 63(3), 791-817 (2017). 2. Wu, L., Wang, D., Evans, J. A.: Large teams develop and small teams disrupt science and technology. Nature, 566(7744), 378 (2019). 3. Azoulay, P.: Small research teams ‘disrupt’science more radically than large ones. Retrieved from nature.com/articles/d41586-019-00350-3  (2019) 4. Bornmann, L., Tekles, A.: Disruptive papers published in Scientometrics. Scientometrics, 1-6 (2019). 5. Meirmans, S., Butlin, R. K., Charmantier, A., Engelstädter, J., Groot, A. T., King, K. C., Neiman, M.: Science policies: How should science funding be allocated? An evolutionary biologists’ perspective. Journal of Evolutionary Biology. Jun 18 (2019). 6. Li, J., Yin, Y., Fortunato, S., Wang, D.: A dataset of publication records for nobel laure-ates. Scientific Data, 6(1) (2019). 7. Chan, H. F., Önder, A. S., Torgler, B.. The first cut is the deepest: repeated interactions of coauthorship and academic productivity in Nobel laureate teams. Scientometrics, 106(2), 509-524 (2016). 8. Clarivate Analytics.: Clarivate Analytics Reveals Annual Forecast of Future Nobel Prize Recipients. Retrieved from https://clarivate.com/news/clarivate-analytics-reveals-annual-forecast-of-future-nobel-prize-recipients/ (2018).  

https://www.ideals.illinois.edu/bitstream/2142/106575/1/Contribution_477_final.pdf

Nobel-Prize-winning papers are significantly more highly-cited but not more disruptive than non-prize-winning counterparts

Abstract

Similar works

Full text

Available Versions

IDEALS @ Illinois

Illinois Digital Environment for Access to Learning and Scholarship Repository