Coverage of highly-cited documents in Google Scholar, Web of Science, and Scopus: a multidisciplinary comparison

This study explores the extent to which bibliometric indicators based on counts of highly-cited documents could be affected by the choice of data source. The initial hypothesis is that databases that rely on journal selection criteria for their document coverage may not necessarily provide an accurate representation of highly-cited documents across all subject areas, while inclusive databases, which give each document the chance to stand on its own merits, might be better suited to identify highly-cited documents. To test this hypothesis, an analysis of 2,515 highly-cited documents published in 2006 that Google Scholar displays in its Classic Papers product is carried out at the level of broad subject categories, checking whether these documents are also covered in Web of Science and Scopus, and whether the citation counts offered by the different sources are similar. The results show that a large fraction of highly-cited documents in the Social Sciences and Humanities (8.6%-28.2%) are invisible to Web of Science and Scopus. In the Natural, Life, and Health Sciences the proportion of missing highly-cited documents in Web of Science and Scopus is much lower. Furthermore, in all areas, Spearman correlation coefficients of citation counts in Google Scholar, as compared to Web of Science and Scopus citation counts, are remarkably strong (.83-.99). The main conclusion is that the data about highly-cited documents available in the inclusive database Google Scholar does indeed reveal significant coverage deficiencies in Web of Science and Scopus in several areas of research. Therefore, using these selective databases to compute bibliometric indicators based on counts of highly-cited documents might produce biased assessments in poorly covered areas.Alberto Martín-Martín enjoys a four-year doctoral fellowship (FPU2013/05863) granted by the Ministerio de Educación, Cultura, y Deportes (Spain)

Reviewing, indicating, and counting books for modern research evaluation systems

In this chapter, we focus on the specialists who have helped to improve the conditions for book assessments in research evaluation exercises, with empirically based data and insights supporting their greater integration. Our review highlights the research carried out by four types of expert communities, referred to as the monitors, the subject classifiers, the indexers and the indicator constructionists. Many challenges lie ahead for scholars affiliated with these communities, particularly the latter three. By acknowledging their unique, yet interrelated roles, we show where the greatest potential is for both quantitative and qualitative indicator advancements in book-inclusive evaluation systems.Comment: Forthcoming in Glanzel, W., Moed, H.F., Schmoch U., Thelwall, M. (2018). Springer Handbook of Science and Technology Indicators. Springer Some corrections made in subsection 'Publisher prestige or quality

The Prestige of Social Scientists in Spain and France: An Examination of Their h-Index Values Using Scopus and Google Scholar

We analyze the prestige of 1,500 scholars in economics, sociology, and management who have Spanish and French institutional affiliations operationalized by their h-index in Scopus and Google Scholar. We use a negative binomial count model to examine how some individual factors affect the h-index from both databases. The results show a non-monotonic relationship between the researchers’ career length and their h-index. There is a positive and statistically significant relationship between total research output and the h-index. The share of publications in English over total publications has also a positive and statistically significant effect on the h-index, except in a single case, while the share of publications in other foreign languages does not have such effect. Finally, we found that the effects of the number of citations received by documents in English (international impact) and by those in the vernacular language (local or regional impact) on the h-index vary according to the database, the country, and the discipline in questio

Long-distance interdisciplinarity leads to higher scientific impact

Scholarly collaborations across disparate scientific disciplines are challenging. Collaborators are likely to have their offices in another building, attend different conferences, and publish in other venues; they might speak a different scientific language and value an alien scientific culture. This paper presents a detailed analysis of success and failure of interdisciplinary papers—as manifested in the citations they receive. For 9.2 million interdisciplinary research papers published between 2000 and 2012 we show that the majority (69.9%) of co-cited interdisciplinary pairs are “win-win” relationships, i.e., papers that cite them have higher citation impact and there are as few as 3.3% “lose-lose” relationships. Papers citing references from subdisciplines positioned far apart (in the conceptual space of the UCSD map of science) attract the highest relative citation counts. The findings support the assumption that interdisciplinary research is more successful and leads to results greater than the sum of its disciplinary parts

Characterization of the health Portuguese scientific production on Scopus

Introdução – É particularmente importante uma avaliação objetiva e imparcial da qualidade da atividade científica dos investigadores, mas também do comportamento científico e pedagógico das instituições de ensino superior, as quais, por sua vez, são o espelho do empenho de um país na sua componente de I&D. Objetivo – O presente estudo bibliométrico pretende caracterizar a presença portuguesa na Scopus e analisar a produção científica portuguesa classificada na área da saúde e indexada nesta base de dados. Métodos – Analisou-se a produção científica portuguesa referente ao período de janeiro de 2000 a dezembro de 2015. A abordagem centrou-se nas seguintes variáveis: categorias de classificação da Scopus; tipologia de documentos indexados; títulos de revistas; autores; distribuição por anos de publicação; afiliação institucional e países de origem dos autores com quem foram estabelecidas relações de parceria científica. Consideraram-se três grandes categorias de classificação na Scopus (Life Sciences, Health Sciences e Social Sciences & Humanities, usando filtros temáticos), porque a área da saúde tanto assume componentes exatas como transversais. Conjugou-se o descritor Portugal com a modalidade affiliation country. Os dados foram alinhados pela terminologia das variáveis em estudo (affiliation, author, country, doctype, source, subject, year) e fundidos num só ficheiro por variável. Resultados – A Scopus contempla 198.749 resultados com afiliação em Portugal. Na área da saúde contabilizaram-se, no total, 71.232 trabalhos, o que significa uma percentagem de 35,8%. Estes encontram-se distribuídos pelos três grupos de classificação: Health Sciences (59,1%), Life Sciences (34%) e Social Sciences & Humanities (6,9%). O artigo original (78,1%) consubstancia a forma mais usada pelos autores portugueses para a divulgação dos resultados de investigação, logo seguido do artigo de revisão (8,9%), dos paper (3,9%) e das letter (3,1%). Os últimos cinco anos são os mais representativos na produção científica (58,4%). Analisando as revistas onde os investigadores portugueses mais publicam, constata-se que são portuguesas sete das primeiras dez. A maioria da produção científica com visibilidade internacional é oriunda das universidades, sendo a Universidade do Porto a que mais se destaca. A parceria científica com outros investigadores destaca a colaboração nacional, mas também com os Estados Unidos, Espanha, Reino Unido, Alemanha, França, Itália, Países Baixos e Brasil, por esta ordem. De destacar que é a Universidade de São Paulo (no Brasil) a maior instituição parceira com 788 trabalhos. Discussão e Conclusões – Na informação da área da saúde indexada na Scopus, as universidades desempenham um papel fundamental, destacando-se a Universidade do Porto. Também os índices de coautoria e sobretudo a colaboração internacional com investigadores de outras nacionalidades têm aumentado ao longo dos anos. Os benefícios e os méritos desta colaboração internacional ao nível da investigação incluem a partilha e a transferência de conhecimento e equipamento, associando os investigadores a uma grande rede científica, bem como o acelerar do processo de investigação, aumentando a visibilidade dos artigos. A produção científica portuguesa da saúde evidencia a existência de vínculos com diversos países, produto das parcerias, dos projetos globais e dos financiamentos.ABSTRACT - Introduction – Evaluation of the quality of the scientific research as well as the scientific and teaching attitude of the higher education institutions is of paramount importance since it reflects the R&D landscape in a country. Objective – This bibliometric study aims at characterizing the Portuguese sample in Scopus and analyzing the Portuguese contribution to the scientific outputs indexed in this database on Health. Methods – The scientific outputs produced in Portugal between January 2000 and December 2015 were analyzed. The study focused on the following variables: Scopus classification categories, type of indexed documents, journal titles, authors, publication year, affiliation, country of origin of collaborators. Three major Scopus categories were considered (Life Sciences, Health Sciences and Social Sciences & Humanities) as the healthcare area can assume both exact and transversal components. The description Portugal was conjugated with category affiliation country. Data was aligned according to the variables considered in this study (affiliation, author, country, doctype, source, subject, and year) to produce a single file per variable. Results – Scopus includes 198,749 results with Portuguese affiliation. In the health sector, a total of 71,232 outputs were found which represents 35.8%. These outputs fall into the three categories considered as follows: Health Sciences (59.1%), Life Sciences (34%) and Social Sciences & Humanities (6.9%). Original paper (78.1%) is the most common type of document used by Portuguese authors to communicate research results, followed by review paper (8.9%), paper (3.9%) and letter (3.1%). The past 5 years are the most representative in terms of scientific outputs (58.4%). By analyzing the journals in which Portuguese authors publish the most, we found that 7 out of 10 are Portuguese journals. Most of scientific outputs published in international journals come from universities being the Oporto University the most prolific. In terms of scientific collaborations, national partnerships are the most common but there are also collaborations with scientists from other countries such USA, Spain, UK, Germany, France, Italy, Netherlands and Brazil, in this order. Noteworthy, Universidade de São Paulo in Brazil is the institution with which Portuguese scientists have produced more scientific outputs: 788. Discussion & Conclusions – Universities and, in particular, Oporto University, are major contributors for the health information indexed in Scopus. Also, the collaboration index, specially, the international collaboration, has risen in the last few years. The benefits of these collaborations include sharing and transferring knowledge and equipment which greatly contributes to the widespread dissemination of research and accelerates scientific discovery. Portuguese health scientific outputs reveal close collaborations with other countries through partnerships, global projects and funding

Social Gerontology- Integrative and Territorial Aspects: A Citation Analysis of Subject Scatter and Database Coverage

To determine the mix of resources used in social gerontology research, a citation analysis was conducted. A representative sample of citations was selected from three prominent gerontology journals and information was added to determine subject scatter and database coverage for the cited materials. Results indicate that a significant portion of gerontology research, even from a social science perspective, relies roughly equally on medical resources as it does social science resources. Furthermore, there is a small but defined core of literature constituting scholarly “territory” unique to gerontology. Analysis of database indexing indicated that broad, interdisciplinary databases provide more comprehensive coverage of the cited materials than do subject-specific databases