The h-index

The h-index is a mainstream bibliometric indicator, since it is widely used in academia, research management and research policy. While its advantages have been highlighted, such as its simple calculation, it has also received widespread criticism. The criticism is mainly based on the negative effects it may have on scholars, when the index is used to describe the quality of a scholar. The “h” means “highly-cited” and “high achievement”, and should not be confused with the last name of its inventor, Hirsch. Put simply, the h-index combines a measure of quantity and impact in a single indicator. Several initiatives try to provide alternatives to the h-index to counter some of its shortcomings

The h-index

The h-index is a mainstream bibliometric indicator, since it is widely used in academia, research management and research policy. While its advantages have been highlighted, such as its simple calculation, it has also received widespread criticism. The criticism is mainly based on the negative effects it may have on scholars, when the index is used to describe the quality of a scholar. The “h” means “highly-cited” and “high achievement”, and should not be confused with the last name of its inventor, Hirsch. Put simply, the h-index combines a measure of quantity and impact in a single indicator. Several initiatives try to provide alternatives to the h-index to counter some of its shortcomings

The generalized propensity score methodology for estimating unbiased journal impact factors

The journal impact factor (JIF) proposed by Garfield in the year 1955 is one of the most commonly used and prominent citation-based indicators of the performance and significance of a scientific journal. The JIF is simple, reasonable, clearly defined, and comparable over time and, what is more, can be easily calculated from data provided by Thomson Reuters, but at the expense of serious technical and methodological flaws. The paper discusses one of the core problems: The JIF is affected by bias factors (e.g., document type) that have nothing to do with the prestige or quality of a journal. For solving this problem, we suggest using the generalized propensity score methodology based on the Rubin Causal Model. Citation data for papers of all journals in the ISI subject category "Microscopy” (Journal Citation Report) are used to illustrate the proposa

What does impact mean for grantees? Cultural consensus in perceived personal, organizational and societal impacts of small-scale funding initiatives of the Volkswagen Foundation

Compared to large-scale funding programs, small-scale ones receive little attention in research. For instance, the funding initiatives of the Volkswagen Foundation (VWS) generates impetus for scientific developments. In small-scale funding programs the knowledge of fellows as informants of funding programs and their impacts can be bet-ter considered. With Cultural Consensus Theory and the notion of an “impact culture” a methodological frame-work will be suggested for evaluation purposes of small-scale funding programs. “Impact culture” is defined as the shared knowledge about perceived impacts of a funding program among fellows. In the study commissioned by the VWS two-stage surveys with different levels of impacts were conducted (individual, institutional, societal) for two funding initiatives of the VWS. One single impact culture of each initiative can be found with consistent response set across all levels

Scientific analysis of data on proposals and the decision-making procedure of the FWF with particular focus on the programme "Stand-Alone Projects" in the years 2010-2019

his study follows on from the analyses of the Austrian Science Fund's (FWF) decision-making procedures for the years 1999-2009. The new study analyses the FWF Stand-Alone Projects programme for the years 2010-2019. The data basis is: 50 board meetings 10,871 funding applications 23,646 international reviews 1,582 final reports 1,317 final report reviews Based on this data, further detailed analyses will follow in the coming months and years

Cross-disciplinary research: What configurations of fields of science are found in grant proposals today?

Considering the complexity of the world problems, it seems evident that they do not fit straightforwardly into a disciplinary framework. In this context, the question arises as to whether and how frequently several disciplines cooperate on research projects. Cross-disciplinary cooperation in research might be difficult for two reasons. On one hand, many researchers feel that efforts to achieve methodological rigour, exactness, and control are only possible in the circumscribed area of a discipline. On the other hand, it is claimed that funding organizations, with their rigid disciplinary classification systems, impede cross-disciplinary research in the context of their selection and evaluation procedures. For a total of N = 8,496 grant proposals submitted to the Austrian Science Fund (FWF) from 1999 to 2009, detailed codings of the subdisciplines involved were available for the statistical analysis. Latent class analysis produced 12 latent classes or configurations of fields of science. Mono-disciplinary projects are very well represented in physics/astronomy/mechanics, geosciences, and clinical medicine. Cross-disciplinarity is found particularly in research project proposals of fields of science with clearly overlapping content (e.g. preclinical and clinical medicine) and mainly in research proposals submitted by fields of science within the humanities and social science

The influence of the applicants' gender on the modeling of a peer review process by using latent Markov models

In the grant peer review process we can distinguish various evaluation stages in which assessors judge applications on a rating scale. Bornmann & al. [2008] show that latent Markov models offer a fundamentally good opportunity to model statistically peer review processes. The main objective of this short communication is to test the influence of the applicants' gender on the modeling of a peer review process by using latent Markov models. We found differences in transition probabilities from one stage to the other for applications for a doctoral fellowship submitted by male and female applicant

Types of research output profiles: A multilevel latent class analysis of the Austrian Science Fund's final project report data

Starting out from a broad concept of research output, this article looks at the question as to what research outputs can typically be expected from certain disciplines. Based on a secondary analysis of data from final project reports (ex post research evaluation) at the Austrian Science Fund (FWF), Austria's central funding organization for basic research, the goals are (1) to find, across all scientific disciplines, types of funded research projects with similar research output profiles; and (2) to classify the scientific disciplines in homogeneous segments bottom-up according to the frequency distribution of these research output profiles. The data comprised 1,742 completed, FWF-funded research projects across 22 scientific disciplines. The multilevel latent class (LC) analysis produced four LCs or types of research output profiles: ‘Not Book', ‘Book and Non-Reviewed Journal Article', ‘Multiple Outputs', and ‘Journal Article, Conference Contribution, and Career Development'. The class membership can be predicted by three covariates: project duration, requested grant sum, and project head's age. In addition, five segments of disciplines can be distinguished: ‘Life Sciences and Medicine', ‘Social Sciences/Arts and Humanities', ‘Formal Sciences', ‘Technical Sciences', and ‘Physical Sciences'. In ‘Social Sciences/Arts and Humanities' almost all projects are of the type ‘Book and Non-Reviewed Journal Article', but, vice versa, not all projects of the ‘Book and Non-reviewed Journal Article' type are in the ‘Social Sciences/Arts and Humanities' segment. The research projects differ not only qualitatively in their output profile; they also differ quantitatively, so that projects can be ranked according to amount of outpu

Are there any frontiers of research performance? Efficiency measurement of funded research projects with the Bayesian stochastic frontier analysis for count data

In recent years, scientometrics has devoted increasing attention to the question of measurement of productivity and efficiency in research. In econometrics, the question is usually examined using data envelopment analysis. Alternatively, in this paper we propose using a statistical approach, Bayesian stochastic frontier analysis (B-SFA), that explicitly considers the stochastic nature of (count) data. The Austrian Science Fund (FWF) made data available to us from their peer review process (ex-ante peer evaluation of proposals, final research product reports) and bibliometric data. The data analysis was done for a subsample of N = 1,046 FWF-funded projects (in Life Science and Medicine, Formal and Physical Sciences). For two outcome variables, a general latent research product dimension (CFACTOR) and the total number of publications (P), technical efficiency values (TE) were estimated for each project using the SFA production functions. The TE values for CFACTOR and P were on average 0.86 and 0.27, as compared with a maximum TE value of 1.0. With regard to CFACTOR, female PIs, younger PIs, and projects with longer durations have slightly higher TE than male PIs, older PIs, and projects with shorter durations. A simulation study showed the statistical behavior of the procedure under different sampling conditions