Citation gaming induced by bibliometric evaluation: A country-level comparative analysis

It is several years since national research evaluation systems around the globe started making use of quantitative indicators to measure the performance of researchers. Nevertheless, the effects on these systems on the behavior of the evaluated researchers are still largely unknown. For investigating this topic, we propose a new inwardness indicator able to gauge the degree of scientific self-referentiality of a country. Inwardness is defined as the proportion of citations coming from the country over the total number of citations gathered by the country. A comparative analysis of the trends for the G10 countries in the years 2000-2016 reveals a net increase of the Italian inwardness. Italy became, both globally and for a large majority of the research fields, the country with the highest inwardness and the lowest rate of international collaborations. The change in the Italian trend occurs in the years following the introduction in 2011 of national regulations in which key passages of professional careers are governed by bibliometric indicators. A most likely explanation of the peculiar Italian trend is a generalized strategic use of citations in the Italian scientific community, both in the form of strategic author self-citations and of citation clubs. We argue that the Italian case offers crucial insights on the constitutive effects of evaluation systems. As such, it could become a paradigmatic case in the debate about the use of indicators in science-policy contexts. © 2019 Baccini et al

Citation gaming induced by bibliometric evaluation: A country-level comparative analysis

It is several years since national research evaluation systems around the globe started making use of quantitative indicators to measure the performance of researchers. Nevertheless, the effects on these systems on the behavior of the evaluated researchers are still largely unknown. For investigating this topic, we propose a new inwardness indicator able to gauge the degree of scientific self-referentiality of a country. Inwardness is defined as the proportion of citations coming from the country over the total number of citations gathered by the country. A comparative analysis of the trends for the G10 countries in the years 2000-2016 reveals a net increase of the Italian inwardness. Italy became, both globally and for a large majority of the research fields, the country with the highest inwardness and the lowest rate of international collaborations. The change in the Italian trend occurs in the years following the introduction in 2011 of national regulations in which key passages of professional careers are governed by bibliometric indicators. A most likely explanation of the peculiar Italian trend is a generalized strategic use of citations in the Italian scientific community, both in the form of strategic author self-citations and of citation clubs. We argue that the Italian case offers crucial insights on the constitutive effects of evaluation systems. As such, it could become a paradigmatic case in the debate about the use of indicators in science-policy contexts

Clustering Time-Series Gene Expression Data Using Smoothing Spline Derivatives

Microarray data acquired during time-course experiments allow the temporal variations in gene expression to be monitored. An original postprandial fasting experiment was conducted in the mouse and the expression of 200 genes was monitored with a dedicated macroarray at 11 time points between 0 and 72 hours of fasting. The aim of this study was to provide a relevant clustering of gene expression temporal profiles. This was achieved by focusing on the shapes of the curves rather than on the absolute level of expression. Actually, we combined spline smoothing and first derivative computation with hierarchical and partitioning clustering. A heuristic approach was proposed to tune the spline smoothing parameter using both statistical and biological considerations. Clusters are illustrated a posteriori through principal component analysis and heatmap visualization. Most results were found to be in agreement with the literature on the effects of fasting on the mouse liver and provide promising directions for future biological investigations

Effect of probiotics on oral candidiasis: A systematic review and meta-analysis

Oral candidiasis (OC) is an increasing health problem due to the introduction of new drugs, population aging, and increasing prevalence of chronic illness. This study systematically reviews the effects of the oral intake of probiotics, prebiotics, and synbiotics on Candida spp. counts (colony-forming units (CFU)/mL) in oral and palatal samples. A literature search was conducted. Twelve studies, eight randomized clinical trials (RCTs), and four pre-post studies, resulted as eligible for the meta-analysis, which was performed through a Bayesian random-effects model. All studies analyzed probiotics, and none of them analyzed prebiotics or synbiotics. The treatments effects were measured in terms of odds ratio (OR) of OC (CFU/mL >102, 103, or 104). The meta-analytic OR was 0.71 (95% credibility interval (CrI): 0.37, 1.32), indicating a beneficial effect of treatment; the I2 index was 56.3%. Focusing only on RCTs, the OR was larger and more precise at 0.53 (95% CrI: 0.27, 0.93). The effect of treatment appeared to be larger on denture wearers. Our findings indicate that the intake of probiotics can have a beneficial effect on OC and that the effects could vary according to the patients’ characteristics. Due to the presence of medium–high-risk studies, the results should be interpreted with caution

Incremental Material Flow Analysis with Bayesian Inference

Material Flow Analysis (MFA) is widely used to study the life-cycles of materials from production, through use, to reuse, recycling or disposal, in order to identify environmental impacts and opportunities to address them. However, development of this type of analysis is often constrained by limited data, which may be uncertain, contradictory, missing or over-aggregated. This article proposes a Bayesian approach, in which uncertain knowledge about material flows is described by probability distributions. If little data is initially available, the model predictions will be rather vague. As new data is acquired, it is systematically incorporated to reduce the level of uncertainty. After reviewing previous approaches to uncertainty in MFA, the Bayesian approach is introduced, and a general recipe for its application to Material Flow Analysis is developed. This is applied to map global production of steel, using Markov Chain Monte Carlo simulations. As well as aiding the analyst, who can get started in the face of incomplete data, this incremental approach to MFA also supports efforts to improve communication of results by transparently accounting for uncertainty throughout.ngineering and Physical Sciences Research Council. Grant Numbers: EP/K039326/1, EP/N02351x/

Peer review and citation data in predicting university rankings, a large-scale analysis

Most Performance-based Research Funding Systems (PRFS) draw on peer review and bibliometric indicators, two different method- ologies which are sometimes combined. A common argument against the use of indicators in such research evaluation exercises is their low corre- lation at the article level with peer review judgments. In this study, we analyse 191,000 papers from 154 higher education institutes which were peer reviewed in a national research evaluation exercise. We combine these data with 6.95 million citations to the original papers. We show that when citation-based indicators are applied at the institutional or departmental level, rather than at the level of individual papers, surpris- ingly large correlations with peer review judgments can be observed, up to r <= 0.802, n = 37, p < 0.001 for some disciplines. In our evaluation of ranking prediction performance based on citation data, we show we can reduce the mean rank prediction error by 25% compared to previous work. This suggests that citation-based indicators are sufficiently aligned with peer review results at the institutional level to be used to lessen the overall burden of peer review on national evaluation exercises leading to considerable cost savings

Missing not at random in end of life care studies : multiple imputation and sensitivity analysis on data from the ACTION study

Background: Missing data are common in end-of-life care studies, but there is still relatively little exploration of which is the best method to deal with them, and, in particular, if the missing at random (MAR) assumption is valid or missing not at random (MNAR) mechanisms should be assumed. In this paper we investigated this issue through a sensitivity analysis within the ACTION study, a multicenter cluster randomized controlled trial testing advance care planning in patients with advanced lung or colorectal cancer. Methods: Multiple imputation procedures under MAR and MNAR assumptions were implemented. Possible violation of the MAR assumption was addressed with reference to variables measuring quality of life and symptoms. The MNAR model assumed that patients with worse health were more likely to have missing questionnaires, making a distinction between single missing items, which were assumed to satisfy the MAR assumption, and missing values due to completely missing questionnaire for which a MNAR mechanism was hypothesized. We explored the sensitivity to possible departures from MAR on gender differences between key indicators and on simple correlations. Results: Up to 39% of follow-up data were missing. Results under MAR reflected that missingness was related to poorer health status. Correlations between variables, although very small, changed according to the imputation method, as well as the differences in scores by gender, indicating a certain sensitivity of the results to the violation of the MAR assumption. Conclusions: The findings confirmed the importance of undertaking this kind of analysis in end-of-life care studies