Measuring research performance in international collaboration

Chinchilla-Rodríguez, Zaida; Miguel, Sandra; Perianes-Rodríguez, Antonio. (2016). Measuring research performance in international collaboration. 14th International Congress of Information, Info '2016. La Havana, Cuba, October 31- November 4, 2016.International collaboration in the creation of knowledge is responsible to change the structural stratification of science having profound implications for the governance of science. Analysis of collaboration in Latin American and Caribbean countries is of particular significance, because initiatives are often the result of “research-for-aid” arrangements, generally based on North–South asymmetries. However, collaboration for mutual benefit and excellence has gained increasing acceptance, with “partner” selection becoming a strategic priority to enhance one’s own production. The general aim of this study is to quantify the benefit rate in visibility and impact of scientific production in the field of nanoscience and nanotechnology (NST) bearing in mind the different types of output (total, in leadership, excellent, and excellent with leadership) of the six main producers of knowledge in NST in Latin America in the period 2003-2013. More specifically we aspire to visualize the networks of international collaboration in a given country (ego-network) to represent the difference between the citations received per type of output, and identify the associates with whom a country has greater potential and capacity to generate knowledge of high quality, as well as the differences existing in terms of visibility depending on the type of production analyzed. In short, we wish to determine the benefits of such collaborative efforts. In this way we could respond to questions such as: a) With which countries is collaboration established? and b) With which collaborating countries are the greatest volume of citations per document obtained, according to the type of output.This work was made possible through financing by the Project NANOMETRICS (Ref. CSO2014-57770-R) supported by Ministerio de Economía y Competitividad of SpainPeer reviewe

Follow the leader : on the relationship between leadership and scholarly impact in international collaborations

National contributions to science are influenced by a number of factors, including economic capacity, national scientific priorities, science policy, and institutional settings and cultures. Nations do not have equal opportunities to access the global scientific market, and therefore, often seek out international partners with complementary resources and expertise. This study aims at investigating national collaboration strategies, with a focus on research leadership—measured through corresponding authorship—and its relationship with scientific impact. Results show that countries with higher R&D investments are more scientifically independent, and confirm that international collaboration is positively related to citation impact. However, leadership in international collaboration is inversely related with a countries’ share of international collaboration and there is a very little relationship between citation impact and international leadership. For instance, most countries—and particularly those that have fewer resources—have higher scientific impact when they are not leading. This suggests that, despite increasing global participation in science, most international collaborations are asymmetrical, and that the research system remains structured around a few dominate nations

How to Combine Research Guarantor and Collaboration Patterns to Measure Scientific Performance of Countries in Scientific Fields: Nanoscience and Nanotechnology as a Case Study

This paper presents a comparative benchmarking of scientometric indicators to characterize the patterns of publication and research performance at the country level, in a specific field (nanoscience and nanotechnology) during the period 2003–2013. The aim was to assess how decisive collaboration may be in attaining a sound level of scientific performance, and how important leadership is for publication. To this end, we used a new methodological approach that contributes to the debate about scientific autonomy or dependency of countries in their scientific performance, and which may serve as an aid in decision-making with regard to research management. The results reveal that in terms of output, USA and China are the main producers; and due to the huge increase in their publications, Iran, India, and Australia can be considered emerging countries. The results highlight USA, Ireland, and Singapore as the countries with the highest levels of normalized citation impact, scientific excellence, and good management of leadership, all of which suggest strong scientific development and scientific autonomy. Also worth mentioning is the high visibility and scientific consolidation of China and Australia, despite the meager growth of their output. Moreover, the performance results indicate that in most cases the countries whose pattern of publication is more international tend to have greater visibility. Yet, a high degree of leadership does not always translate as a high performance level; the contrary is often true. Due to the limitations of the sample and characteristics of the field, we propose that future studies evaluate the generation of new knowledge in this field and refine the approach presented here, so as to better measure scientific performance.Projects I + D + I, State Programme of Research, Development and Innovation oriented to the Challenges of the Society: NANOMETRICS (Ref. CSO2014-57770-R) supported by Ministerio de Economía y Competitividad of Spain.Peer reviewedPeer Reviewe

Argentine research on agroindustry and its international collaboration (2007-2016)

Este trabajo tiene como objetivo comparar los patrones de colaboración de la producción científica argentina de agroindustria de acuerdo con el alcance geográfico de la investigación, es decir, temas dirigidos a lectores nacionales y temas que van más allá de la frontera nacional recogida en la base de datos Scopus para el periodo 2007-2016. El comportamiento de ambos grupos se analiza y describe de la siguiente manera: a) se discrimina según se trate de temas de investigación globales o locales, aplicando el criterio de ausencia o presencia de topónimos nacionales, respectivamente; b) se presenta el volumen de coautoría con diferentes países agrupados por continentes ; c) se determinan las 20 palabras claves más frecuentes para cada caso y, d) se pone en relación con el Índice-H de los países, utilizado en este caso como indicador de impacto científico del país colaborador. Los principales resultados muestran que solo con Europa se logra la proporción de colaboración a los niveles de la ciencia del país (40%). Con países de continentes como Oceanía, Asia y África, la colaboración es muy minoritaria (5% o menos). La distribución en temas globales y locales se mantiene en todos los continentes con leves incrementos en Europa, América del Norte y Caribe y Oceanía a favor de los temas locales. La correlación entre la colaboración internacional y el impacto científico que tienen los países con los que se colabora es relativamente positiva. Los temas sobre los que se investiga se anclan principalmente en la Patagonia, el litoral marítimo y se centran especialmente en los bovinos y el maíz. El estudio aporta información útil para pensar la conformación de equipos de investigación, la planificación de programas de movilidad internacional y alianzas estratégicas en relación con la colaboración futura, y lo hace en un área estratégica que genera fuertes expectativas para el desarrollo económico del país.This work aims to compare the collaboration patterns of the Argentine scientific production of Agroindustry according to the geographical scope of the research, that is, topics addressed to national readers and topics that go beyond the national border. Scopus database were used to carry out this study which span for the period 2007-2016. Collaboration patterns of both groups were analyzed and described as follows: a) whether they are global or local research topics, applying the criterion of absence or presence of national place names, respectively; b) the volume of co-authorship with different countries grouped by continents is presented; c) the 20 most frequent keywords are determined for each case and, d) the H-Index of countries was used as an indicator of the scientific impact of the collaborating country. The main results show that European countries are the main collaborator in Agroindustry research with more than 40% of co-authored papers, while Oceania, Asia, and Africa present lower collaboration rates (5% or less). The distribution of papers in global and local topics appear in all continents with slight increases in Europe, North America, and the Caribbean and Oceania in favor of local topics. The correlation between international collaboration and the scientific impact of the countries with which Argentina collaborates is relatively positive. The topics investigated are mainly anchored in Patagonia, the maritime coastline, and are especially focused on cattle and corn. The study provides useful information to think about the formation of research teams, the planning of international mobility programs, and strategic alliances for future collaboration, and it does so in a strategic area that generates strong expectations for the economic development of the country.Dossier: Estudios métricos de la información: abordajes teóricos, metodológicos y empíricosFacultad de Humanidades y Ciencias de la Educació

Citation Flows in the Zones of Influence of Scientific

Domestic citation to papers from the same country and the greater citation impact of documents involving international collaboration are two phenomena that have been extensively studied and contrasted. Here, however, we showthat it is not somuch a national bias,but that papers have a greater impact on their immediate environments, an impact that is diluted as that environment grows. For this reason, the greatest biases are observed in countries with a limited production. Papers that involve international collaboration have a greater impact in general, on the one hand, because they have multiple “immediate environments,” and on the other because of their greater quality or prestige. In short, one can say that science knows no frontiers. Certainly there is a greater impact on the authors’ immediate environment, but this does not necessarily have to coincide with their national environments, which fade in importance as the collaborative environment expands

An empirical review of the different variants of the Probabilistic Affinity Index as applied to scientific collaboration

Responsible indicators are crucial for research assessment and monitoring. Transparency and accuracy of indicators are required to make research assessment fair and ensure reproducibility. However, sometimes it is difficult to conduct or replicate studies based on indicators due to the lack of transparency in conceptualization and operationalization. In this paper, we review the different variants of the Probabilistic Affinity Index (PAI), considering both the conceptual and empirical underpinnings. We begin with a review of the historical development of the indicator and the different alternatives proposed. To demonstrate the utility of the indicator, we demonstrate the application of PAI to identifying preferred partners in scientific collaboration. A streamlined procedure is provided, to demonstrate the variations and appropriate calculations. We then compare the results of implementation for five specific countries involved in international scientific collaboration. Despite the different proposals on its calculation, we do not observe large differences between the PAI variants, particularly with respect to country size. As with any indicator, the selection of a particular variant is dependent on the research question. To facilitate appropriate use, we provide recommendations for the use of the indicator given specific contexts.Comment: 35 pages, 3 figures, 5 table

Investigación española en células madre (1997-2010). Localización y evolución de las principales líneas de investigación a través de los KeyWords Plus

•Objective: To render and analyze the evolution and dynamics of the main research lines in the Spanish scientific output on stem cells between 1997 and 2010. •Material and Methods: We retrieved the documents from the Science Citation Index (SCI). The units of analysis were the descriptors KeyWords Plus (KW +). The tools used for visualization have been software Pajek combined with Pathfinder Network (PFnet), and VOSviewer software. This study analyzes the period includes between 1997-2007, and the evolution of the research into three sub-periods: 1997-2001, 2002-2006 and 2007-2010. •Results and Discussion: The results of the full period locate up to six main research lines. In the first period, there is a strong presence of the descriptors that represent documents related to hematology and oncology. However, the other research lines do not begin to be clearly detected until the second period. Here,we can locate clusters of descriptors related to hematopoietic stem cell research, others related with the generation, proliferation and differentiation of stem cells and finally, alternate associated to an emerging cluster of neural progenitor cells. The results show that the weight of these latter groups make them more evident in the last period. •Conclusion: The visualization of the relationships between the KW + has yielded two complementary images of the situation and evolution of Spanish research on stem cells. The methodology has identified areas of research both consolidated and emerging, intuiting the development of a thematic domain over time. The visualization software complement each other quite well, matching in the identification of the main research lines and in the location of the most influential descriptors

SPECIAL TRACK: Present and future of research metadata: where do we want to go from here?

To steer the scientific system towards specific goals, it is first necessary to develop an effective understanding of all phases and aspects of the research workflow. Research metadata, as the collective record of traces that are generated when scientific activities take place, serves as evidence of these activities. Therefore, the availability of authoritative research metadata is essential for science-related decision-making at various levels. In the past, large-scale research metadata collections mostly dealt with items in the public record, such as bibliographic metadata about academic publications. There used to be few of these large-scale metadata collections, and they were often provided by commercial actors, those which invested the necessary resources to compile and process disperse public information with the goal of turning it into usable services. As the capabilities of available technologies increase, each day more sectors of the scientific system are becoming aware of how their activities could benefit from updating their workflows, a process often referred to as digital transformation. Thus, a plethora of tools and standards are being developed to streamline processes, increase interoperability, and in general overcome the limitations of the paper era. This is having a large effect in the quantity and quality of research metadata that is now being recorded. A clear example of the above is the case of bibliographic metadata. Currently, an increasing number of organizations, spurred by the decreasing barriers to collecting and processing large amounts of bibliographic metadata, are already providing services and datasets that rival the offerings of the traditional commercial providers. Some of these new datasets, provided under open licenses that allow unrestricted reuse and redistribution, have boosted innovation by allowing the development of downstream applications that rely on these metadata collections. However, as scientific activities in general and scientific communication in particular are increasingly moving to the digital space, traditional bibliographic metadata is no longer the only kind of research metadata that is being collected and processed at a large scale to inform decisions. Social network platforms now capture a portion of academic-related conversations and other kinds of interactions. Processes such as peer review that were previously carried out behind closed doors are now being opened, generating their own public trace. Publishing platforms are implementing increasingly sophisticated methods to track and mine user actions for their benefit. All these recent developments call for a discussion on the role of research metadata in the scientific system going forward. This discussion should be open to a large variety of stakeholders, including data providers, scientometricians, academic librarians, higher education institutions, policy managers, and developers of downstream applications. The topics of the contributions to this special track can include: • Analyses of the suitability of research metadata sources for specific use cases • Sustainability and governance of research metadata • Innovations in research metadata • Downstream applications of open research metadata • Surveillance through research metadata Contributions to this special track would be open to everyone interested and peer-reviewed. The format of the session would be 15-20 minutes per presentation, with time for questions after each presentation

Patrones de comunicación e impacto de la producción científica cubana en salud pública

[EN] Objective: to characterize the Cuban pattern of scientific communication in public health in Scopus database on the basis of the output and collaboration patterns and their influence on the impact of publications.Methods: bibliometric indicators of output, visibility and collaboration taken from SCImago Institutions Rankings and SCImago Journal and Country Rank portals were used, according to Scopus database, in the Public Health, Environmental and Occupational Health category in 2003-2011 period.Results: the communication pattern showed an increasing tendency of the scientific output, with great leadership of Cuban authors and poor national and international collaboration. This increase did not have a higher impact on the international community; 7.22 % of documents were published in high impact journals and 2.16 % were among excellence documents. The excellence output with leadership was almost non-existent. Seventy five percent of the output was seen in low impact journals (fourth quartile) and most of it in national journals. The English output accounted for less than 30% of the total amount but had higher impact than the Spanish articles.Conclusions: the pattern of Cuban scientific communication in public health was characterized, in which low number of English publications, of publications in high impact journals and poor national and international collaborations are factors that may influence on the low scientific impact. These results can be used to supplement the assessment of research in public health within the tenth basic function of this specialty.[ES] Objetivo: caracterizar el patrón cubano de comunicación científica en salud pública en la base de datos Scopus a partir de los patrones de producción y colaboración y su influencia en el impacto de las publicaciones.Métodos: se aplicaron indicadores bibliométricos de producción, visibilidad y colaboración extraídos de los portales SCImago Institutions Rankings y SCImago Journal and Country Rank a partir de datos de Scopus, categoría Public Health, Environmental and Occupational Health, período 2003-2011.Resultados: el patrón de comunicación presentó una tendencia al incremento de la producción científica con un alto liderazgo de autores cubanos y escasa colaboración nacional e internacional. Este incremento no repercutió en un mayor impacto en la comunidad internacional; el 7,22 % de los documentos aparecieron en revistas de alto impacto, y el 2,16 % entre los documentos de excelencia. La producción de excelencia con liderazgo fue casi inexistente. El 75 % de la producción se ubicó en revistas de bajo impacto (cuarto cuartil) y la mayoría en revistas nacionales. La producción en inglés representó menos del 30 % del total y alcanzaron mayor impacto que los artículos en español.Conclusiones: se caracterizó el patrón de comunicación científica cubano en salud pública, donde la poca publicación en inglés, en revistas de alto impacto y la insuficiente colaboración nacional e internacional son factores que pudieran estar influenciando en el bajo impacto científico. Estos resultados pueden utilizarse como complemento de la evaluación de la investigación en salud pública en el marco de su décima función esencial. Palabras clave: salud pública, evaluación de la investigación, funciones esenciales de la salud pública, cienciometría, bases de datos, indicadores bibliométricos, colaboración científica, revistas, SCImago Journal and Country Rank, SCImago Institutions Rankings.Peer reviewe

Bibliometric analysis of regional Latin America's scientific output in Public Health through SCImago Journal & Country Rank

Background: In the greater framework of the essential functions of Public Health, our focus is on a systematic, objective, external evaluation of Latin American scientific output, to compare its publications in the area of Public Health with those of other major geographic zones. We aim to describe the regional distribution of output in Public Health, and the level of visibility and specialization, for Latin America; it can then be characterized and compared in the international context. Methods: The primary source of information was the Scopus database, using the category “Public Health, Environmental and Occupational Health”, in the period 1996–2011. Data were obtained through the portal of SCImago Journal and Country Rank. Using a set of qualitative (citation-based), quantitative (document recount) and collaborative (authors from more than one country) indicators, we derived complementary data. The methodology serves as an analytical tool for researchers and scientific policy-makers. Results: The contribution of Latin America to the arsenal of world science lies more or less midway on the international scale in terms of its output and visibility. Revealed as its greatest strengths are the high level of specialization in Public Health and the sustained growth of output. The main limitations identified were a relative decrease in collaboration and low visibility. Conclusions: Collaboration is a key factor behind the development of scientific activity in Latin America. Although this finding can be useful for formulating research policy in Latin American countries, it also underlines the need for further research into patterns of scientific communication in this region, to arrive at more specific recommendations.We acknowledge support of the publication fee by the CSIC Open Access Publication Support Initiative through its Unit of Information Resources for Research (URICI)