How journal rankings can suppress interdisciplinary research. A comparison between Innovation Studies and Business & Management

This study provides quantitative evidence on how the use of journal rankings can disadvantage interdisciplinary research in research evaluations. Using publication and citation data, it compares the degree of interdisciplinarity and the research performance of a number of Innovation Studies units with that of leading Business & Management schools in the UK. On the basis of various mappings and metrics, this study shows that: (i) Innovation Studies units are consistently more interdisciplinary in their research than Business & Management schools; (ii) the top journals in the Association of Business Schools' rankings span a less diverse set of disciplines than lower-ranked journals; (iii) this results in a more favourable assessment of the performance of Business & Management schools, which are more disciplinary-focused. This citation-based analysis challenges the journal ranking-based assessment. In short, the investigation illustrates how ostensibly 'excellence-based' journal rankings exhibit a systematic bias in favour of mono-disciplinary research. The paper concludes with a discussion of implications of these phenomena, in particular how the bias is likely to affect negatively the evaluation and associated financial resourcing of interdisciplinary research organisations, and may result in researchers becoming more compliant with disciplinary authority over time.Comment: 41 pages, 10 figure

How Journal Rankings can suppress Interdisciplinary Research – A Comparison between Innovation Studies and Business & Management

This study provides new quantitative evidence on how journal rankings can disadvantage interdisciplinary research during research evaluations. Using publication data, it compares the degree of interdisciplinarity and the research performance of innovation studies units with business and management schools in the UK. Using various mappings and metrics, this study shows that: (i) innovation studies units are consistently more interdisciplinary than business and management schools; (ii) the top journals in the Association of Business Schools’ rankings span a less diverse set of disciplines than lower ranked journals; (iii) this pattern results in a more favourable performance assessment of the business and management schools, which are more disciplinary-focused. Lastly, it demonstrates how a citation-based analysis challenges the ranking-based assessment. In summary, the investigation illustrates how ostensibly ‘excellence-based’ journal rankings have a systematic bias in favour of mono-disciplinary research. The paper concludes with a discussion of implications of these phenomena, in particular how resulting bias is likely to affect negatively the evaluation and associated financial resourcing of interdisciplinary organisations, and may encourage researchers to be more compliant with disciplinary authority.Interdisciplinary, Evaluation, Ranking, Innovation, Bibliometrics, REF

Will the REF disadvantage interdisciplinary research? The inadvertent effects of journal rankings

A failure to engage in interdisciplinary work risks creating intellectual inbreeding and could push research away from socially complex issues. Ismael Rafols asks why there is a bias against interdisciplinary research, and why the REF will work to suppress an otherwise useful body of research

The structure of the Arts & Humanities Citation Index: A mapping on the basis of aggregated citations among 1,157 journals

Using the Arts & Humanities Citation Index (A&HCI) 2008, we apply mapping techniques previously developed for mapping journal structures in the Science and Social Science Citation Indices. Citation relations among the 110,718 records were aggregated at the level of 1,157 journals specific to the A&HCI, and the journal structures are questioned on whether a cognitive structure can be reconstructed and visualized. Both cosine-normalization (bottom up) and factor analysis (top down) suggest a division into approximately twelve subsets. The relations among these subsets are explored using various visualization techniques. However, we were not able to retrieve this structure using the ISI Subject Categories, including the 25 categories which are specific to the A&HCI. We discuss options for validation such as against the categories of the Humanities Indicators of the American Academy of Arts and Sciences, the panel structure of the European Reference Index for the Humanities (ERIH), and compare our results with the curriculum organization of the Humanities Section of the College of Letters and Sciences of UCLA as an example of institutional organization

Interdisciplinarity and research on local issues: evidence from a developing country

This paper explores the relationship between interdisciplinarity and research pertaining to local issues. Using Colombian publications from 1991 until 2011 in the Web of Science, we investigate the relationship between the degree of interdisciplinarity and the local orientation of the articles. We find that a higher degree of interdisciplinarity in a publication is associated with a greater emphasis on Colombian issues. In particular, our results suggest that research that combines cognitively disparate disciplines, what we refer to as distal interdisciplinarity, tends to be associated with more local focus of research. We discuss the implications of these results in the context of policies aiming to foster the local socio-economic impact of research in developing countries.Comment: 24 page

Interdisciplinarity and research on local issues: evidence from a developing country

This paper examines the role of interdisciplinarity on research pertaining to local issues. Using Colombian publications from 1991 until 2011 in the Web of Science, we investigate the relationship between the degree of interdisciplinarity and the local orientation of the articles. We find that a higher degree of interdisciplinarity in a publication is associated with a greater emphasis on local issues. In particular, our results support the view that research that combines cognitively disparate disciplines, what we refer to as distal interdisciplinarity, is associated with more local focus of research. We discuss the policy implications of these results in the context of national research assessments targeting excellence and socio-economic impact

Assessing the crossdisciplinarity of technology-enhanced learning with science overlay maps and diversity measures

This paper deals with the assessment of the crossdisciplinarity of technology-enhanced learning (TEL). Based on a general discussion of the concept interdisciplinarity and a summary of the discussion in the field two empirical methods from scientometrics are introduced and applied. Science overlay maps and the Rao-Stirling-Diversity index are used to analyze the TEL field with a scientometric analysis. The science overlay maps show that a wide variety of disciplines contribute to research in the field. The analysis reveals that the field has been operating on a relatively high level of crossdisciplinarity in the last 10 years compared to 6 other fields of reference. Only in 2004 a decrease in the level of crossdisciplinarity could be identified