Citations: Indicators of Quality? The Impact Fallacy

We argue that citation is a composed indicator: short-term citations can be considered as currency at the research front, whereas long-term citations can contribute to the codification of knowledge claims into concept symbols. Knowledge claims at the research front are more likely to be transitory and are therefore problematic as indicators of quality. Citation impact studies focus on short-term citation, and therefore tend to measure not epistemic quality, but involvement in current discourses in which contributions are positioned by referencing. We explore this argument using three case studies: (1) citations of the journal Soziale Welt as an example of a venue that tends not to publish papers at a research front, unlike, for example, JACS; (2) Robert Merton as a concept symbol across theories of citation; and (3) the Multi-RPYS ("Multi-Referenced Publication Year Spectroscopy") of the journals Scientometrics, Gene, and Soziale Welt. We show empirically that the measurement of "quality" in terms of citations can further be qualified: short-term citation currency at the research front can be distinguished from longer-term processes of incorporation and codification of knowledge claims into bodies of knowledge. The recently introduced Multi-RPYS can be used to distinguish between short-term and long-term impacts.Comment: accepted for publication in Frontiers in Research Metrics and Analysis; doi: 10.3389/frma.2016.0000

Determinants of the international influence of a R&D organisation: a bibliometric approach

Traditionally, studies on the influence and impact of knowledge-producing organisations have been addressed by means of strict economic analysis, stressing their economic impact to a local, regional or national extent. In the present study, an alternative methodology is put forward in order to evaluate the international scientific impact and influence of a knowledge-producing and -diffusing institution. We introduce a new methodology, based on scientometric and bibliometric tools, which complement traditional assessments by considering the influence of a R&D institution when looking at the scientific production undertaken and the recognition of its relevance by its international peer community. Focusing on the most prolific scientific areas of INESC Porto, and resorting to published scientific work recorded in the Science Citation Index (SCI), we show that INESC Porto has enlarged its international scientific network. The logit estimations demonstrate that the wide geographical influence of INESC Porto scientific research is a result not of its international positioning in terms of co-authorships, but rather a result of the quality of its scientific output.Impact and influence assessment methods; R&D Institutions; Bibliometrics, Scientometrics; knowledge network; INESC Porto

Determinants of the international influence of a R&D organisation: a bibliometric approach

Traditionally, studies on the influence and impact of knowledge-producing organisations have been addressed by means of strict economic analysis, stressing their economic impact to a local, regional or national extent. In the present study, an alternative methodology is put forward in order to evaluate the international scientific impact and influence of a knowledge-producing and -diffusing institution. We introduce a new methodology, based on scientometric and bibliometric tools, which complement traditional assessments by considering the influence of a R&D institution when looking at the scientific production undertaken and the recognition of its relevance by its international peer community. Focusing on the most prolific scientific areas of INESC Porto, and resorting to published scientific work recorded in the Science Citation Index (SCI), we show that INESC Porto has enlarged its international scientific network. The logit estimations demonstrate that the wide geographical influence of INESC Porto scientific research is a result not of its international positioning in terms of co-authorships, but rather a result of the quality of its scientific output.Impact and influence assessment methods; R&D Institutions; Bibliometrics, Scientometrics; knowledge network; INESC Porto

Linking science to technology: using bibliographic references in patents to build linkage schemes.

In this paper, we develop and discuss a method to design a linkage scheme that links the systems of science and technology through the use of patent citation data. After conceptually embedding the linkage scheme in the current literature on science-technology interactions and associations, the methodology and algorithms used to decelop the linkage scheme are discussed in detail. The method is subsequently tested on and applied to subsets of USPTO patents. The results point to highly skewed citation distributions, enabling us to discern between those fields of technology that are highly science-interactive and those fields where technology develoment is highly independent from the scientific literature base.Science; Patents; Systems; Data; Algorithms; Distribution;

Assessing the influence of R&D institutions by mapping international scientific networks: the case of INESC Porto

Although scientometric and bibliometric studies embrace a much wider perspective of the linkages/networks of R&D institutions than standard economic studies, to the best of our knowledge, these studies have not yet made use of scientometric tools to analyse the influence and impact of R&D institutions. Moreover, the international perspective has so far been neglected both in standard and bibliometric studies. Based on networks of 1239 foreign co-authorships and 13035 foreign citation linkages, we demonstrate that INESC Porto international influence has considerably expanded since 2003, a year that coincided with the implementation of an internal policy of granting monetary prizes to publications in scientific international journals. In terms of co-authorship, the network of INESC Porto more than duplicated (13 countries in the initial period to 27 in 2004-07). In terms of citations, INESC Porto’s network encompassed almost 40 countries during the whole period (1996-2007). Its more prolific units (optoelectronics, energy and multimedia) presented a rather distinct pattern both in terms of size and evolution of the corresponding network boundaries. The network size of foreign co-authorships was not much different between the three units by the beginning of the 2000s (around 10 countries) but it evolved quite distinctly. The most remarkable pattern was registered by the multimedia (UTM) unit, whose network size rose exponentially to 21 countries in 2004-07. This contrasted with the decline (down to 8 countries) of the energy (USE) unit. The citation network of the optoelectronic unit (UOSE) was by far the largest, until 2003, involving 34 distinct countries, which contrasted with the size of USE (12 countries) and UTM (1 country). But again, after 2003, the size of the citation network of USE and UTM converged spectacularly to that of UOSE’s, reaching in the last period 21 and 16, respectively. The influence of INESC Porto reaches all five continents, especially when we consider citation networks. Indeed, excluding the citations from authors affiliated in Portuguese institutions, those that most cite INESC Porto’s (and UOSE’s) works are affiliated in institutions located in China, the UK and the US. The scientific works produced by USE influences mostly authors affiliated in institutions located in India, China and Spain, whereas for UTM the corresponding countries are the US, Germany and Italy. We infer from the evidence analysed that not only did the boundaries of INESC Porto’s scientific network substantially enlarge in the period of analysis (1996-2007) but its ‘quality’ also evidenced a positive evolution, with authors affiliated in institutions located in the scientific frontier countries citing works of INESC Porto (and its units).Bibliometrics, Knowledge networks; R&D Institutions

Assessing the influence of R&D institutions by mapping international scientific networks: the case of INESC Porto

Although scientometric and bibliometric studies embrace a much wider perspective of the linkages/networks of R&D institutions than standard economic studies, to the best of our knowledge, these studies have not yet made use of scientometric tools to analyse the influence and impact of R&D institutions. Moreover, the international perspective has so far been neglected both in standard and bibliometric studies. Based on networks of 1239 foreign co-authorships and 13035 foreign citation linkages, we demonstrate that INESC Porto international influence has considerably expanded since 2003, a year that coincided with the implementation of an internal policy of granting monetary prizes to publications in scientific international journals. In terms of co-authorship, the network of INESC Porto more than duplicated (13 countries in the initial period to 27 in 2004-07). In terms of citations, INESC Porto’s network encompassed almost 40 countries during the whole period (1996-2007). Its more prolific units (optoelectronics, energy and multimedia) presented a rather distinct pattern both in terms of size and evolution of the corresponding network boundaries. The network size of foreign co-authorships was not much different between the three units by the beginning of the 2000s (around 10 countries) but it evolved quite distinctly. The most remarkable pattern was registered by the multimedia (UTM) unit, whose network size rose exponentially to 21 countries in 2004-07. This contrasted with the decline (down to 8 countries) of the energy (USE) unit. The citation network of the optoelectronic unit (UOSE) was by far the largest, until 2003, involving 34 distinct countries, which contrasted with the size of USE (12 countries) and UTM (1 country). But again, after 2003, the size of the citation network of USE and UTM converged spectacularly to that of UOSE’s, reaching in the last period 21 and 16, respectively. The influence of INESC Porto reaches all five continents, especially when we consider citation networks. Indeed, excluding the citations from authors affiliated in Portuguese institutions, those that most cite INESC Porto’s (and UOSE’s) works are affiliated in institutions located in China, the UK and the US. The scientific works produced by USE influences mostly authors affiliated in institutions located in India, China and Spain, whereas for UTM the corresponding countries are the US, Germany and Italy. We infer from the evidence analysed that not only did the boundaries of INESC Porto’s scientific network substantially enlarge in the period of analysis (1996-2007) but its ‘quality’ also evidenced a positive evolution, with authors affiliated in institutions located in the scientific frontier countries citing works of INESC Porto (and its units). Length: 64 pagesBibliometrics, Knowledge networks; R&D Institutions