Applying Blockchain Solutions to Address Research Reproducibility and Enable Scientometric Analysis

A worldwide reproducibility crisis around published scientific studies has gained attention from academics, journalists, and concerned citizens in recent decades. The inability to reliably reproduce experiments from scholarly research—especially in areas of high- impact science—has far-reaching social and economic implications. Fraud may seem an obvious culprit, but in our data-intensive world, vague methods, unclear standards, and even accidental mismanagement of digital resources can all be contributing factors. Reproducibility is an area of increasing focus within the scientometrics community and looking to emerging technologies to help mitigate reproducibility challenges makes practical sense. In the Web 3.0 era, the promise of distributed computing, the maturation of cloud services, and other novel convergences point toward new ways to enable bibliometric reproducibility. Concurrently, research artifacts beyond the peer-reviewed article are growing in prominence—datasets, algorithms, pre-prints—all serve an expanding role in research dissemination and discovery. In this paper we present an overview of some new approaches—with particular focus on the benefits of blockchain-based software systems—for managing research information and improving scientometric reproducibility

Exploration of Reproducibility Issues in Scientometric Research Part 2: Conceptual Reproducibility

This is the second part of a small-scale explorative study in an effort to assess reproducibility issues specific to scientometrics research. This effort is motivated by the desire to generate empirical data to inform debates about reproducibility in scientometrics. Rather than attempt to reproduce studies, we explore how we might assess "in principle" reproducibility based on a critical review of the content of published papers. While the first part of the study (Waltman et al. 2018) focuses on direct reproducibility - that is the ability to reproduce the specific evidence produced by an original study using the same data, methods, and procedures, this second part is dedicated to conceptual reproducibility - that is the robustness of knowledge claims towards verification by an alternative approach using different data, methods and procedures. The study is exploratory: it investigates only a very limited number of publications and serves us to develop instruments for identifying potential reproducibility issues of published studies: These are a categorization of study types and a taxonomy of threats to reproducibility. We work with a select sample of five publications in scientometrics covering a variation of study types of theoretical, methodological, and empirical nature. Based on observations made during our exploratory review, we conclude with open questions on how to approach and assess the status of conceptual reproducibility in scientometrics intended for discussion at the special track on "Reproducibility in Scientometrics" at STI2018 in Leiden