Comparing two "thermometers": Impact factors of 20 leading economic journals according to Journal Citation Reports and Scopus

Impact factors for 20 journals ranked first by Journal Citation Reports (JCR) were compared with the same indicator calculated on the basis of citation data obtained from Scopus database. A significant discrepancy was observed as Scopus, though results differed from title to title, found in general more citations than listed in JCR. This also affected ranking of the journals. More thorough examination of two selected titles proved that the divergence resulted mainly from difference in coverage of two products, although other important factors also play their part

With or without h-index? Comparing aggregates of rankings based on seven popular bibliometric indicators

We apply five majority-rule-based ordinal ranking methods to data on economic, management and political science journals in order to produce aggregate journal rankings. First, we calculate aggregates for the set of rankings based on seven popular bibliometric indicators (impact factor, 5-year impact factor, immediacy index, article influence score, h-index, SNIP and SJR). Then, we exclude the Hirsch index and repeat the calculations. We perform the comparative correlation analysis of the aggregates and the initial rankings. We use two rank measures of correlation, Kendall’s tau and the share of coinciding pairs. The analysis demonstrates that aggregate rankings represent the set of single-indicator-based rankings better than any of the seven rankings themselves. Among the single-indicator-based rankings themselves, the best representations of their set are produced by the 5-year impact factor. The least representative are rankings based on the immediacy index. The exclusion of the Hirsch index from the set of indicators does not change these results

Qualitätsanalyse von Zeitschriften in den Wirtschaftswissenschaften – über Zitationsdatenbanken und Impaktfaktoren im Online-Zeitalter

Die Einschätzung der Qualität von wissenschaftlicher Forschung ist sowohl für die Forschungsgemeinde als auch für die Öffentlichkeit wichtig. Trotz der Kritik ist eine quantitative Beurteilung notwendig. In der Wissenschaft spielen die Zitate und die daraus abgeleiteten Impaktfaktoren eine wichtige Rolle. Dieser Artikel stellt drei Zitationsdatenbanken und ihre Funktionsweise vor: das Web of Science, welches den Social Science Citation Index enthält, Google Scholar und RePEc. Darüber hinaus werden die Vor- und Nachteile des Impaktfaktors, einer der wichtigsten Kennzahlen für wissenschaftliche Zeitschriften, herausgearbeitet. Mit Fokus auf die Wirtschaftswissenschaften ist sicherlich das RePEc-Netzwerk das am besten geeignete Forum zur Beurteilung von (quantitativen) Maßzahlen sowohl für Zeitschriften als auch für Working-Paper-Reihen. Jedoch ist die tatsächliche Erfassung der Zitate noch unvollständig, so dass alle Kennzahlen noch als experimentell betrachtet werden sollten.Wirtschaftszeitschrift, Publikationsanalyse, Datenbank, Wirtschaftspolitische Wirkungsanalyse, Deutschland

A Review of Theory and Practice in Scientometrics

Scientometrics is the study of the quantitative aspects of the process of science as a communication system. It is centrally, but not only, concerned with the analysis of citations in the academic literature. In recent years it has come to play a major role in the measurement and evaluation of research performance. In this review we consider: the historical development of scientometrics, sources of citation data, citation metrics and the “laws" of scientometrics, normalisation, journal impact factors and other journal metrics, visualising and mapping science, evaluation and policy, and future developments

A Study of Open Access Publishing by NHMRC Grant Recipients

This study is the first comprehensive study to measure open access compliance under Australia’s earliest national-level open access policy, the National Health and Medical Research Council Open Access Policy. The key findings were that over two-thirds of the journal articles were open access, most were in journals with a small number of postprints located in Australian institutional repositories. This study provides a greater understanding of open access publishing under a funder open access mandate

Altmetrics for Digital Libraries: Concepts, Applications, Evaluation, and Recommendations

The volume of scientific literature is rapidly increasing, which has led to researchers becoming overloaded by the number of articles that they have available for reading and difficulties in estimating their quality and relevance (e.g., based on their research interests). Library portals, in these circumstances, are increasingly getting more relevant by using quality indicators that can help researchers during their research discovery process. Several evaluation methods (e.g., citations, Journal Impact Factor, and peer-reviews) have been used and suggested by library portals to help researchers filter out the relevant articles (e.g., articles that have received high citations) for their needs. However, in some cases, these methods have been criticized, and a number of weaknesses have been identified and discussed. For example, citations usually take a long time to appear, and some articles that are important can remain uncited. With the growing presence of social media today, new alternative indicators, known as “altmetrics,” have been encountered and proposed as complementary indicators to traditional measures (i.e., bibliometrics). They can help to identify the online attention received by articles, which might act as a further indicator for research assessment. One often mentioned advantage of these alternative indicators is, for example, that they appear much faster compared to citations. A large number of studies have explored altmetrics for different disciplines, but few studies have reported about altmetrics in the fields of Economics and Business Studies. Furthermore, no studies can be found so far that analyzed altmetrics within these disciplines with respect to libraries and information overload. Thus, this thesis explores opportunities for introducing altmetrics as new method for filtering relevant articles (in library portals) within the discipline of Economic and Business Studies literature. To achieve this objective, we have worked on four main aspects of investigating altmetrics and altmetrics data, respectively, of which the results can be used to fill the gap in this field of research. (1) We first highlight to what extent altmetric information from the two altmetric providers Mendeley and Altmetric.com is present within the journals of Economics and Business Studies. Based on the coverage, we demonstrate that altmetrics data are sparse in these disciplines, and when considering altmetrics data for real-world applications (e.g., in libraries), higher aggregation levels, such as journal level, can overcome their sparsity well. (2) We perform and discuss the correlations of citations on article and journal levels between different types and sources of altmetrics. We could show that Mendeley counts are positive and strongly correlated with citation counts on both article and journal levels, whereas other indicators such as Twitter counts and Altmetric Attention Score are significantly correlated only on journal level. With these correlations, we could suggest Mendeley counts for Economic and Business Studies journals/articles as an alternative indicator to citations. (3) In conjunction with the findings related to altmetrics in Economics and Business Studies journals, we discuss three use cases derived from three ZBW personas in terms of altmetrics. We investigate the use of altmetrics data for potential users with interests in new trends, social media platforms and journal rankings. (4) We investigated the behavior of economic researchers using a survey by exploring the usefulness of different altmetrics on journal level while they make decisions for selecting one article for reading. According to the user evaluation results, we demonstrate that altmetrics are not well known and understood by the economic community. However, this does not mean that these indicators are not helpful at all to economists. Instead, it brings forward the problem of how to introduce altmetrics to the economic community in the right way using which characteristics (e.g., as visible numbers attached at library records or behind the library’s relevance ranking system). Considering the aforementioned findings of this thesis, we can suggest several forms of presenting altmetric information in library portals, using EconBiz as the proof-of-concept, with the intention to assist both researchers and libraries to identify relevant journals or articles (e.g., highly mentioned online and recently published) for their need and to cope with the information overload

Altmetrics for Digital Libraries

The volume of scientific literature is increasing and researchers have difficulties in estimating their quality and relevance. Library portals, therefore, are getting more relevant by using quality indicators to help researchers during their research process. With the growing presence of social media, altmetrics have been proposed as complementary indicators to traditional measures. Altmetrics can help to identify online attention and can appear much faster than citations. This study explores altmetrics for filtering relevant articles (in library portals) within the discipline of Economic and Business Studies literature. Firstly, it highlights the altmetrics presence from Mendeley and Altmetric.com for the journals in the above-mentioned disciplines. It presents correlations between citation and altmetrics on article and journal level, suggesting Mendeley counts as an alternative indicator to citations. Afterward, it investigates the use of altmetrics data for potential users with interests in new trends, social media platforms, and journal rankings. Lastly, it explores the behavior of economic researchers using a survey by discovering the usefulness of different altmetrics. With the findings of this study, several forms of altmetrics in library portals are discussed, using EconBiz as the proof-of-concept, to assist both researchers and libraries to identify relevant journals or articles and to cope with the information overload