How journal rankings can suppress interdisciplinary research. A comparison between Innovation Studies and Business & Management

This study provides quantitative evidence on how the use of journal rankings can disadvantage interdisciplinary research in research evaluations. Using publication and citation data, it compares the degree of interdisciplinarity and the research performance of a number of Innovation Studies units with that of leading Business & Management schools in the UK. On the basis of various mappings and metrics, this study shows that: (i) Innovation Studies units are consistently more interdisciplinary in their research than Business & Management schools; (ii) the top journals in the Association of Business Schools' rankings span a less diverse set of disciplines than lower-ranked journals; (iii) this results in a more favourable assessment of the performance of Business & Management schools, which are more disciplinary-focused. This citation-based analysis challenges the journal ranking-based assessment. In short, the investigation illustrates how ostensibly 'excellence-based' journal rankings exhibit a systematic bias in favour of mono-disciplinary research. The paper concludes with a discussion of implications of these phenomena, in particular how the bias is likely to affect negatively the evaluation and associated financial resourcing of interdisciplinary research organisations, and may result in researchers becoming more compliant with disciplinary authority over time.Comment: 41 pages, 10 figure

How Journal Rankings can suppress Interdisciplinary Research – A Comparison between Innovation Studies and Business & Management

This study provides new quantitative evidence on how journal rankings can disadvantage interdisciplinary research during research evaluations. Using publication data, it compares the degree of interdisciplinarity and the research performance of innovation studies units with business and management schools in the UK. Using various mappings and metrics, this study shows that: (i) innovation studies units are consistently more interdisciplinary than business and management schools; (ii) the top journals in the Association of Business Schools’ rankings span a less diverse set of disciplines than lower ranked journals; (iii) this pattern results in a more favourable performance assessment of the business and management schools, which are more disciplinary-focused. Lastly, it demonstrates how a citation-based analysis challenges the ranking-based assessment. In summary, the investigation illustrates how ostensibly ‘excellence-based’ journal rankings have a systematic bias in favour of mono-disciplinary research. The paper concludes with a discussion of implications of these phenomena, in particular how resulting bias is likely to affect negatively the evaluation and associated financial resourcing of interdisciplinary organisations, and may encourage researchers to be more compliant with disciplinary authority.Interdisciplinary, Evaluation, Ranking, Innovation, Bibliometrics, REF

A Review of Theory and Practice in Scientometrics

Scientometrics is the study of the quantitative aspects of the process of science as a communication system. It is centrally, but not only, concerned with the analysis of citations in the academic literature. In recent years it has come to play a major role in the measurement and evaluation of research performance. In this review we consider: the historical development of scientometrics, sources of citation data, citation metrics and the “laws" of scientometrics, normalisation, journal impact factors and other journal metrics, visualising and mapping science, evaluation and policy, and future developments

Betweenness and Diversity in Journal Citation Networks as Measures of Interdisciplinarity -- A Tribute to Eugene Garfield --

Journals were central to Eugene Garfield's research interests. Among other things, journals are considered as units of analysis for bibliographic databases such as the Web of Science (WoS) and Scopus. In addition to disciplinary classifications of journals, journal citation patterns span networks across boundaries to variable extents. Using betweenness centrality (BC) and diversity, we elaborate on the question of how to distinguish and rank journals in terms of interdisciplinarity. Interdisciplinarity, however, is difficult to operationalize in the absence of an operational definition of disciplines, the diversity of a unit of analysis is sample-dependent. BC can be considered as a measure of multi-disciplinarity. Diversity of co-citation in a citing document has been considered as an indicator of knowledge integration, but an author can also generate trans-disciplinary--that is, non-disciplined--variation by citing sources from other disciplines. Diversity in the bibliographic coupling among citing documents can analogously be considered as diffusion of knowledge across disciplines. Because the citation networks in the cited direction reflect both structure and variation, diversity in this direction is perhaps the best available measure of interdisciplinarity at the journal level. Furthermore, diversity is based on a summation and can therefore be decomposed, differences among (sub)sets can be tested for statistical significance. In an appendix, a general-purpose routine for measuring diversity in networks is provided

A comparative study on communication structures of Chinese journals in the social sciences

We argue that the communication structures in the Chinese social sciences have not yet been sufficiently reformed. Citation patterns among Chinese domestic journals in three subject areas -- political science and marxism, library and information science, and economics -- are compared with their counterparts internationally. Like their colleagues in the natural and life sciences, Chinese scholars in the social sciences provide fewer references to journal publications than their international counterparts; like their international colleagues, social scientists provide fewer references than natural sciences. The resulting citation networks, therefore, are sparse. Nevertheless, the citation structures clearly suggest that the Chinese social sciences are far less specialized in terms of disciplinary delineations than their international counterparts. Marxism studies are more established than political science in China. In terms of the impact of the Chinese political system on academic fields, disciplines closely related to the political system are less specialized than those weakly related. In the discussion section, we explore reasons that may cause the current stagnation and provide policy recommendations

Fixing Rule 702: The PCAST Report and Steps to Ensure the Reliability of Forensic Feature-Comparison Methods in the Criminal Courts

In response to PCAST’s recommendation, the Standing Advisory Committee on Evidence Rules convened a meeting on forensic expert testimony, Daubert, and Rule 702 on October 27, 2017, at Boston College Law School to inform itself about the issues.22 The meeting included presentations by twenty-six speakers (including myself) and discussion among the attendees. The purpose of this Article is to summarize aspects of the PCAST report relevant to its recommendation to the Standing Advisory Committee on Evidence Rules and to propose a path forward with respect to Rule 702

The Knowledge Bases of the World's Largest Pharmaceuticals Groups: what do Patent Citations to Non-Patent Literature Reveal?

This paper examines the knowledge bases of the world's largest pharmaceuticals groups by sales. It puts forward the concepts of knowledge breadth and depth as the relevant dimensions along which knowledge bases can be mapped. Breadth is studied by analysing the evolution of specialisation by scientific field over time. It hints at the widening range of bodies of scientific and technological knowledge relevant to firms' innovative activities. Depth (or integration) is studied by analysing the evolution of specialisation across different typologies of research. It hints at the complex, non-linear interdependencies that link the scientific and technological domains. We develop the analyses on the strength of an original database of 33,127 EPO patents, and of 41,931 'non patent document' citations (of which 19,494 were identified as scientific articles included in the ISI databases). The groups studied seem to have incrementally increased the breadth of their knowledge bases, moving toward the fields proper of the new bio-pharmaceutical research trajectory. At the same time, some of the groups studied exhibit remarkable depth in knowledge specialisation in particular fields such as biotechnology, biochemical research and neurosciences. Finally, this paper also provides a first methodological test of possible problems deriving from the use of 'unidentified' patent citations (i.e. added by the examiners together with those proposed by the inventor). We compare a random sample of these citations with a sample of citations explicitly added by the original inventor, and compare the results in terms of scientific specialisations.Knowledge Breadth, Depth, Integration, Patent Citations, Scientific Publications, Pharmaceuticals