Academic serials (especially peer-reviewed journals) play a very critical role in the scientific ecosystem and both integrity and independence are perceived as essential for good editorial governance (Rynes, 2006). For being responsible for articles selection (Bedeian et al., 2009; Feldman, 2008), elite board membership ensures the scientific quality of publications and “occupy key roles as opinion formers, gatekeepers and arbiters of disciplinary values” (Burgess and Shaw, 2010, p.630). So far, board elites have not been subject to a scrutiny proportional to their decision power (Burgess & Shaw, 2010) and an overall lack of transparency has been reported about the general editorial process despite of being actual gatekeepers (Miner, 2003; see also Bedeian et al., 2009; Horan et al., 1993).info:eu-repo/semantics/publishedVersio

Mendonça, S.

Santos, A. T.

English

Repositório Institucional do ISCTE-IUL

1537Microbiology, Neuroscience, Physics, Social Sciences, Technology, and Zoology. Results Which journals are most commonly referenced? The main journals of editorial recommendations from Nature and Science are typical high impact journals (e.g., PNAS and Cell; see Figure 1). The highest share of Nature’s “Research Highlights” refers to publications in Science, whereas Science’s “Editor’s Choice” fails to show the reverse pattern.     Figure 1. Percentage of referenced journals in editorial recommendations.    Figure 2. Percentage of authorship countries in editorial recommendations. Which authorship countries are most common? Geographical distribution of authorship was calculated as fractional percentage on a whole-normalized country-based level and was similarly distributed in both journals (see Figure 2). USA-based scientists author the main share of referenced publications in the editorial recommendations of Nature and Science, preceding EU-based ones and a group of scientists from countries other than the listed ones.  Which research areas are most common? The distribution of editorial recommendations across research areas was similar in Nature and Science, apart from a few exceptions (see Table 1). Table 1. Diverging research areas for editorial recommendations Journal Biology Chemistry Ecology Zoology Nature 9.5 % 4.5 % 14.2 % 6.7 % Science 17.1 % 7.5 % 10.3 % 0.7 %  Outlook The current poster aims at describing the main characteristics of editorial recommendations from two major science journals. We showed similarities and differences regarding the referenced publications, their authorship countries, and their main areas of research. Next steps of analysis entail the systematic comparison of editorial recommendations with concurrent measures of research impact like citation metrics and internet-based mentions. In sum, the current project provides a first description of outstanding publications highlighted in two major science journals, providing a promising new approach to evaluating research output.  Acknowledgments We are grateful to Dr. Alexis-Michel MUGABUSHAKA for helpful comments and discussions about this work. References Aksnes, D. W., Langfeldt, L., & Wouters, P. (2019). Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories. SAGE Open. https://doi.org/10.1177/2158244019829575 Mugabushaka, A.-M, Sadat, Costa Dantas Faria, J. (2020). In Search of Outstanding Research Advances: Prototyping the creation of an open dataset of “editorial highlights”. arXiv:2011.07910 Sadat, J., & Mugabushaka, A.-M. (2020). In Search of Outstanding Research Advances – Exploring Editorial Recommendations. Zenodo. http://doi.org/10.5281/zenodo.4155204 Watching over innovation studies: Profiling the gatekeepers Ana Teresa Santos1 and Sandro Mendonça2 1 atmss@iscte-iul.pt Instituto Universitário de Lisboa (ISCTE-IUL) – Business Research Unit (BRU-IUL), Lisbon (Portugal) 2 sfm@iscte-iul.pt Instituto Universitário de Lisboa (ISCTE-IUL) – Business Research Unit (BRU-IUL), Lisbon (Portugal)  Introduction Academic serials (especially peer-reviewed journals) play a very critical role in the scientific ecosystem and both integrity and independence are perceived as essential for good editorial governance (Rynes, 2006). For being responsible for articles selection (Bedeian et al., 2009; Feldman, 2008), elite board membership ensures the scientific quality of publications and “occupy key roles as opinion formers, gatekeepers and arbiters of disciplinary values” (Burgess and Shaw, 2010, p.630). So far, board elites have not been subject to a scrutiny proportional to their decision power (Burgess & Shaw, 2010) and an overall lack of transparency has been reported about the general editorial process despite of being actual gatekeepers (Miner, 2003; see also Bedeian et al., 2009; Horan et al., 1993).   In this work, we draw on the existing, but limited, body of knowledge available to examine empirically journal editorial boards (EB) in innovation studies field. We shed some light on demographics characteristics of editorial members. We believe this work may add some relevant information for those interested in science governance and social structures of research activities.  The Boards of “Innovation Studies” Fagerberg et al. (2012) studied both the most productive journals in innovation as well as those using the most findings already published. For the 20 top-tier journals identified, we aim to present the demographic features of scholars behind. Scholars’ names, affiliation country and gender were collected from official journal’s editorial page. A total of 3,005 available seats were recorded occupied by 2,440 distinct persons from 30 countries. The size of EBs The mean size of Boards is 150 editors. Although R D Manag Board was found with only 19 editors, others have over 300 scholars: the likes of North-American outlets as Acad Manage J and Acad Manage Rev while Manage Sci stands out as the one with the largest team, 399 editors as presented in Figure 1.   Figure 1: Editorial Board size per journal intercepted by the mean number of editors. Geographies of editorship Braun (2004) defined international journals as those with scholars from 5 countries in EB. Thus, all outlets from our study set are international ones, as their editors come from 8 to 30 different countries. Figure 2 shows a world representation of editors’ frequency found in each country. Darker the colour, higher the frequency of editors.    Figure 2. Geographical location of editorial members. 1538The US overwhelming dominance of membership is clear. García-Carpintero et al. (2010) reminded most science publishing houses are US-based. Europe and India host a significant number of editors. With exception of Australia, nations from the South represent a negligible role in this editorial process with very few editors involved. Gender balance With the purpose of understanding gender balance, we matched gender proportion in ten countries with higher numbers of editors as illustrated in Figure 3.    Figure 3: Number of editors affiliated to the top 10 countries, by gender. US number of editors is by far different from the other countries with around 1,000 men and 400 women editors. However, US are not gender-balanced. Actually, all countries show a higher frequency of memberships held by men editors than women. Metz & Harzing (2009) pointed women’s presence in academia is not long enough to reach levels of seniority which are associated with Board membership. Conclusions The present study is an attempt to understand the demographic structure of innovation EB members which showed to be diverse quantitative and qualitatively. Despite of being an eclectic topic, a low diversity for gender (male predominance) and country affiliation (high representation of US and UK editors) is perceived. A greater diversity is important for EB meet their missions (Jagsi et al., 2008) and a key driver in academia for knowledge development by applying different methodologies and paradigms (Robinson & Dechant, 1997).  References Bedeian, A. G., van Fleet, D. D. & Hyman, H. H. (2009). Scientific Achievement and Editorial Board Membership. Organizational Research Methods, 12(2), 211-238. https://doi.org/10.1177/1094428107309312 Braun, T. (2004). Keeping the Gates of Science Journals. In Handbook of Quantitative Science and Technology Research (pp. 95-114). Springer Netherlands. https://doi.org/10.1007/1-4020-2755-9_5 Burgess, T. F. & Shaw, N. E. (2010). Editorial Board Membership of Management and Business Journals: A Social Network Analysis Study of the Financial Times 40. British Journal of Management, 21(3), 627-648. https://doi.org/10.1111/j.1467-8551.2010.00701.x Fagerberg, J., Fosaas, M. & Sapprasert, K. (2012). Innovation: Exploring the knowledge base. Research Policy, 41(7), 1132-1153. https://doi.org/10.1016/J.RESPOL.2012.03.008 Feldman, D. C. (2008). Building and Maintaining a Strong Editorial Board and Cadre of Ad Hoc Reviewers. In Opening the Black Box of Editorship (pp. 68-74). Palgrave Macmillan UK. https://doi.org/10.1057/9780230582590_7 García-Carpintero, E., Granadino, B. & Plaza, L. (2010). The representation of nationalities on the editorial boards of international journals and the promotion of the scientific output of the same countries. Scientometrics, 84(3), 799-811. https://doi.org/10.1007/s11192-010-0199-3 Horan, J. J., Weber, W. L., Fitzsimmons, P., Maglio, C. J. & Hanish, C. (1993). Further Manifestations of the MOMM Phenomenon. The Counseling Psychologist, 21(2), 278-287. https://doi.org/10.1177/0011000093212011 Jagsi, R., Tarbell, N. J., Henault, L. E., Chang, Y. & Hylek, E. M. (2008). The representation of women on the editorial boards of major medical journals: A 35-year perspective. Archives of Internal Medicine, 168(5), 544-548. https://doi.org/10.1001/archinte.168.5.544 Metz, I. & Harzing, A.-W. (2009). Gender Diversity in Editorial Boards of Management Journals. Academy of Management Learning & Education, 8(4), 540-557. https://doi.org/10.5465/amle.8.4.zqr540 Miner, J. B. (2003). Commentary on Arthur Bedeian’s “The Manuscript Review Process: The Proper Roles of Authors, Referees, and Editors.” Journal of Management Inquiry, 12(4), 339-343. https://doi.org/10.1177/1056492603259056 Robinson, G. & Dechant, K. (1997). Building a Business Case for Diversity. The Academy of Management Executive, 11(3), 21-31. https://doi.org/10.2307/4165408 Rynes, S. L. (2006). “Getting on board” with AMJ: Balancing quality and innovation in the review process. Academy of Management Journal, 49(6), 1097-1102. https://doi.org/10.5465/AMJ.2006.23478050   Keyword Distance Ratio:  Evaluating Keyword Assignment with Word Embeddings Brandon Sepulvado  sepulvado-brandon@norc.org NORC at the University of Chicago, 4350 East West Highway, Bethesda, MD 20814 (USA)  Introduction Despite the emergence of natural language processing (NLP) assisted search within certain new bibliometric databases (Wang et al., 2020), keywords remain a staple of information curation and retrieval within most major bibliometric sources, such as Web of Science, Scopus, and PubMed, and authors still choose keywords or phrases to summarize the information their documents contain. As such, keywords are an institutionalized component of the publication and literature search process, yet how does one empirically evaluate the keywords chosen for a document? This text proposes the Keyword Distance Ratio (KDR) to respond to this question. Theoretically, keywords should, at the same time, be closely related to the content within a document while also remaining sufficiently distinct so as not to convey redundant information. The KDR is a document-level measure that relies upon the Relaxed Word Mover’s Distance (RWMD; Kusner, Sun, Kolkin & Weinberger, 2015) to  quantify these intuitions.  In investigating the utility of the KDR, this text compares article keywords submitted by authors to those chosen by Scopus in its index terms and then illustrates a dramatic difference in document summary based upon the keyword source. Keyword Distance Ratio Relaxed Word Mover’s Distance The KDR relies upon Word Mover’s Distance (WMD; Kusner et al., 2015) in order to obtain distances between keywords and the documents they describe. For an intuitive understanding of WMD, imagine that a document is a cluster of points in n-dimensional space. The points represent the embeddings for words (i.e., keywords and those in the article), and a word’s location is based upon the word’s embedding vector. The current analyses use 300-dimension FastText embeddings (Mikolov, Chen, Corrado & Dean, 2013), which means that each word would be placed in a 300-dimension space. One plots the keywords and words in the document, and the distance then is a function of the effort it takes to move one set of points to the closest points in the other set. To reduce computational complexity, the RWMD relaxes a couple constraints imposed upon the WMD.  Keyword Distance Ratio The KDR entails calculating two sets of RWMDs: (1) between each pair of keywords and (2) between each keyword and its corresponding document. The KDR then compares the sum of each type of distance, adjusting for the number of keywords listed. Let the KDR of document d be:  𝐾𝐾𝐾𝐾𝑅𝑅𝑑𝑑 =  ∑ ∑ 𝑅𝑅𝑅𝑅𝑅𝑅𝐾𝐾𝑘𝑘𝑖𝑖,𝑑𝑑,𝑘𝑘𝑗𝑗,𝑑𝑑𝑖𝑖>𝑗𝑗𝑛𝑛−1𝑗𝑗=1∑ 𝑅𝑅𝑅𝑅𝑅𝑅𝐾𝐾𝑘𝑘𝑖𝑖,𝑑𝑑,𝑎𝑎𝑑𝑑𝑛𝑛𝑖𝑖=1   ⋅ 2(𝑛𝑛 − 1)  In this equation, the numerator sums all pairwise RWMDs between keywords 𝑘𝑘𝑖𝑖 and 𝑘𝑘𝑗𝑗 used to signify the content of scholarly document d; the denominator sums the pairwise RWMDs between keyword 𝑘𝑘𝑖𝑖 and abstract 𝑎𝑎 for the document 𝑑𝑑. This value however will increase automatically with each additional keyword because there are 𝑛𝑛(𝑛𝑛−1)2   possible comparisons in the numerator and only n comparisons in the denominator, where n is the number of keywords. As such, the second part of the KDR equation accounts for the number of keywords. To rephrase more intuitively, the KDR is the ratio of the sum of pairwise RWMDs for all keywords listed for a document to the sum of all the RWMDs between a document's keywords and its abstract; this value is then scaled to account for the number of keywords in a document.   Data This text uses a small corpus of neuroethics articles to demonstrate the utility of the KDR for comparing author-provided keywords to Scopus-assigned keywords (i.e., index terms). The data used come from Scopus, which is an ideal bibliographic database because it has good journal coverage in both the humanities and health-related sciences and indexes more journals than other databases, such as the Web of Science (Falagas, Pitsouni, Malietzis & Pappas, 2008). A keyword-based query was used to obtain publication records from Scopus because this approach has proven successful for neuroethics (Leefmann, Levallois & Hildt, 2016). I searched Scopus for any articles published in English that contain “neuroethic*” in the title, abstract, and/or keyword fields. After excluding articles with missing abstracts and/or keywords, there are 727 publications. In calculating the distance from 

Watching over innovation studies: Profiling the gatekeepers

https://repositorio.iscte-iul.pt/bitstream/10071/24456/1/conferenceobject_84171.pdf

Watching over innovation studies: Profiling the gatekeepers

Abstract

Similar works

Full text

Available Versions

Repositório Institucional do ISCTE-IUL