Universality of Performance Indicators based on Citation and Reference Counts

We find evidence for the universality of two relative bibliometric indicators of the quality of individual scientific publications taken from different data sets. One of these is a new index that considers both citation and reference counts. We demonstrate this universality for relatively well cited publications from a single institute, grouped by year of publication and by faculty or by department. We show similar behaviour in publications submitted to the arXiv e-print archive, grouped by year of submission and by sub-archive. We also find that for reasonably well cited papers this distribution is well fitted by a lognormal with a variance of around 1.3 which is consistent with the results of Radicchi, Fortunato, and Castellano (2008). Our work demonstrates that comparisons can be made between publications from different disciplines and publication dates, regardless of their citation count and without expensive access to the whole world-wide citation graph. Further, it shows that averages of the logarithm of such relative bibliometric indices deal with the issue of long tails and avoid the need for statistics based on lengthy ranking procedures.Comment: 15 pages, 14 figures, 11 pages of supplementary material. Submitted to Scientometric

A reverse engineering approach to the suppression of citation biases reveals universal properties of citation distributions

The large amount of information contained in bibliographic databases has recently boosted the use of citations, and other indicators based on citation numbers, as tools for the quantitative assessment of scientific research. Citations counts are often interpreted as proxies for the scientific influence of papers, journals, scholars, and institutions. However, a rigorous and scientifically grounded methodology for a correct use of citation counts is still missing. In particular, cross-disciplinary comparisons in terms of raw citation counts systematically favors scientific disciplines with higher citation and publication rates. Here we perform an exhaustive study of the citation patterns of millions of papers, and derive a simple transformation of citation counts able to suppress the disproportionate citation counts among scientific domains. We find that the transformation is well described by a power-law function, and that the parameter values of the transformation are typical features of each scientific discipline. Universal properties of citation patterns descend therefore from the fact that citation distributions for papers in a specific field are all part of the same family of univariate distributions.Comment: 9 pages, 6 figures. Supporting information files available at http://filrad.homelinux.or

Comparing research investment to United Kingdom institutions and published outputs for tuberculosis, HIV and malaria: A systematic analysis across 1997-2013

Background: The "Unfinished Agenda" of infectious diseases is of great importance to policymakers and research funding agencies that require ongoing research evidence on their effective management. Journal publications help effectively share and disseminate research results to inform policy and practice. We assess research investments to United Kingdom institutions in HIV, tuberculosis and malaria, and analyse these by numbers of publications and citations and by disease and type of science. Methods: Information on infection-related research investments awarded to United Kingdom institutions across 1997-2010 were sourced from funding agencies and individually categorised by disease and type of science. Publications were sourced from the Scopus database via keyword searches and filtered to include only publications relating to human disease and containing a United Kingdom-based first and/or last author. Data were matched by disease and type of science categories. Investment (United Kingdom pounds) and publications were compared to generate an 'investment per publication' metric; similarly, an 'investment per citation' metric was also developed as a measure of the usefulness of research. Results: Total research investment for all three diseases was £1.4 billion, and was greatest for HIV (£651.4 million), followed by malaria (£518.7 million) and tuberculosis (£239.1 million). There were 17,271 included publications, with 9,322 for HIV, 4,451 for malaria, and 3,498 for tuberculosis. HIV publications received the most citations (254,949), followed by malaria (148,559) and tuberculosis (100,244). According to UK pound per publication, tuberculosis (£50,691) appeared the most productive for investment, compared to HIV (£61,971) and malaria (£94,483). By type of science, public health research was most productive for HIV (£27,296) and tuberculosis (£22,273), while phase I-III trials were most productive for malaria (£60,491). According to UK pound per citation, tuberculosis (£1,797) was the most productive area for investment, compared to HIV (£2,265) and malaria (£2,834). Public health research was the most productive type of science for HIV (£2,265) and tuberculosis (£1,797), whereas phase I-III trials were most productive for malaria (£1,713). Conclusions: When comparing total publications and citations with research investment to United Kingdom institutions, tuberculosis research appears to perform best in terms of efficiency. There were more public health-related publications and citations for HIV and tuberculosis than other types of science. These findings demonstrate the diversity of research funding and outputs, and provide new evidence to inform research investment strategies for policymakers, funders, academic institutions, and healthcare organizations.Infectious Disease Research Networ

Good practices for a literature survey are not followed by authors while preparing scientific manuscripts

The number of citations received by authors in scientific journals has become a major parameter to assess individual researchers and the journals themselves through the impact factor. A fair assessment therefore requires that the criteria for selecting references in a given manuscript should be unbiased with respect to the authors or the journals cited. In this paper, we advocate that authors should follow two mandatory principles to select papers (later reflected in the list of references) while studying the literature for a given research: i) consider similarity of content with the topics investigated, lest very related work should be reproduced or ignored; ii) perform a systematic search over the network of citations including seminal or very related papers. We use formalisms of complex networks for two datasets of papers from the arXiv repository to show that neither of these two criteria is fulfilled in practice

Towards a new crown indicator: an empirical analysis

We present an empirical comparison between two normalization mechanisms for citation-based indicators of research performance. These mechanisms aim to normalize citation counts for the field and the year in which a publication was published. One mechanism is applied in the current so-called crown indicator of our institute. The other mechanism is applied in the new crown indicator that our institute is currently exploring. We find that at high aggregation levels, such as at the level of large research institutions or at the level of countries, the differences between the two mechanisms are very small. At lower aggregation levels, such as at the level of research groups or at the level of journals, the differences between the two mechanisms are somewhat larger. We pay special attention to the way in which recent publications are handled. These publications typically have very low citation counts and should therefore be handled with special care

Does the Committee Peer Review Select the Best Applicants for Funding? An Investigation of the Selection Process for Two European Molecular Biology Organization Programmes

Does peer review fulfill its declared objective of identifying the best science and the best scientists? In order to answer this question we analyzed the Long-Term Fellowship and the Young Investigator programmes of the European Molecular Biology Organization. Both programmes aim to identify and support the best post doctoral fellows and young group leaders in the life sciences. We checked the association between the selection decisions and the scientific performance of the applicants. Our study involved publication and citation data for 668 applicants to the Long-Term Fellowship programme from the year 1998 (130 approved, 538 rejected) and 297 applicants to the Young Investigator programme (39 approved and 258 rejected applicants) from the years 2001 and 2002. If quantity and impact of research publications are used as a criterion for scientific achievement, the results of (zero-truncated) negative binomial models show that the peer review process indeed selects scientists who perform on a higher level than the rejected ones subsequent to application. We determined the extent of errors due to over-estimation (type I errors) and under-estimation (type 2 errors) of future scientific performance. Our statistical analyses point out that between 26% and 48% of the decisions made to award or reject an application show one of both error types. Even though for a part of the applicants, the selection committee did not correctly estimate the applicant's future performance, the results show a statistically significant association between selection decisions and the applicants' scientific achievements, if quantity and impact of research publications are used as a criterion for scientific achievement

Understanding the evolution of immune genes in jawed vertebrates

Driven by co-evolution with pathogens, host immunity continuously adapts to optimize defence against pathogens within a given environment. Recent advances in genetics, genomics and transcriptomics have enabled a more detailed investigation into how immunogenetic variation shapes the diversity of immune responses seen across domestic and wild animal species. However, a deeper understanding of the diverse molecular mechanisms that shape immunity within and among species is still needed to gain insight into-and generate evolutionary hypotheses on-the ultimate drivers of immunological differences. Here, we discuss current advances in our understanding of molecular evolution underpinning jawed vertebrate immunity. First, we introduce the immunome concept, a framework for characterizing genes involved in immune defence from a comparative perspective, then we outline how immune genes of interest can be identified. Second, we focus on how different selection modes are observed acting across groups of immune genes and propose hypotheses to explain these differences. We then provide an overview of the approaches used so far to study the evolutionary heterogeneity of immune genes on macro and microevolutionary scales. Finally, we discuss some of the current evidence as to how specific pathogens affect the evolution of different groups of immune genes. This review results from the collective discussion on the current key challenges in evolutionary immunology conducted at the ESEB 2021 Online Satellite Symposium: Molecular evolution of the vertebrate immune system, from the lab to natural populations

Microwave studies of the fractional Josephson effect in HgTe-based Josephson junctions

The rise of topological phases of matter is strongly connected to their potential to host Majorana bound states, a powerful ingredient in the search for a robust, topologically protected, quantum information processing. In order to produce such states, a method of choice is to induce superconductivity in topological insulators. The engineering of the interplay between superconductivity and the electronic properties of a topological insulator is a challenging task and it is consequently very important to understand the physics of simple superconducting devices such as Josephson junctions, in which new topological properties are expected to emerge. In this article, we review recent experiments investigating topological superconductivity in topological insulators, using microwave excitation and detection techniques. More precisely, we have fabricated and studied topological Josephson junctions made of HgTe weak links in contact with two Al or Nb contacts. In such devices, we have observed two signatures of the fractional Josephson effect, which is expected to emerge from topologically-protected gapless Andreev bound states. We first recall the theoretical background on topological Josephson junctions, then move to the experimental observations. Then, we assess the topological origin of the observed features and conclude with an outlook towards more advanced microwave spectroscopy experiments, currently under development.Comment: Lectures given at the San Sebastian Topological Matter School 2017, published in "Topological Matter. Springer Series in Solid-State Sciences, vol 190. Springer

Harmonic Allocation of Authorship Credit: Source-Level Correction of Bibliometric Bias Assures Accurate Publication and Citation Analysis

Authorship credit for multi-authored scientific publications is routinely allocated either by issuing full publication credit repeatedly to all coauthors, or by dividing one credit equally among all coauthors. The ensuing inflationary and equalizing biases distort derived bibliometric measures of merit by systematically benefiting secondary authors at the expense of primary authors. Here I show how harmonic counting, which allocates credit according to authorship rank and the number of coauthors, provides simultaneous source-level correction for both biases as well as accommodating further decoding of byline information. I also demonstrate large and erratic effects of counting bias on the original h-index, and show how the harmonic version of the h-index provides unbiased bibliometric ranking of scientific merit while retaining the original's essential simplicity, transparency and intended fairness. Harmonic decoding of byline information resolves the conundrum of authorship credit allocation by providing a simple recipe for source-level correction of inflationary and equalizing bias. Harmonic counting could also offer unrivalled accuracy in automated assessments of scientific productivity, impact and achievement