First UK Reproducibility Network Annual Report (2023-2024)

Il s\u27agit du rapport annuel 2023-2024 de l\u27UK Reproducibility Network (UKRN) au sujet de la Science Ouverte avec des exemples anglais de projets inspirant très avancé

Reproducibility in Management Science

With the help of more than 700 reviewers we assess the reproducibility of nearly 500 articles published in the journal Management Science before and after the introduction of a new Data and Code Disclosure policy in 2019. When considering only articles for which data accessibility and hard- and software requirements were not an obstacle for reviewers, the results of more than 95% of articles under the new disclosure policy could be fully or largely computationally reproduced. However, for 29% of articles at least part of the dataset was not accessible to the reviewer. Considering all articles in our sample reduces the share of reproduced articles to 68%. These figures represent a significant increase compared to the period before the introduction of the disclosure policy, where only 12% of articles voluntarily provided replication materials, out of which 55% could be (largely) reproduced. Substantial heterogeneity in reproducibility rates across different fields is mainly driven by differences in dataset accessibility. Other reasons for unsuccessful reproduction attempts include missing code, unresolvable code errors, weak or missing documentation, but also soft- and hardware requirements and code complexity. Our findings highlight the importance of journal code and data disclosure policies, and suggest potential avenues for enhancing their effectiveness

Reforms to improve reproducibility and quality must be coordinated across the research ecosystem: the view from the UKRN Local Network Leads

Many disciplines are facing a “reproducibility crisis”, which has precipitated much discussion about how to improve research integrity, reproducibility, and transparency. A unified effort across all sectors, levels, and stages of the research ecosystem is needed to coordinate goals and reforms that focus on open and transparent research practices. Promoting a more positive incentive culture for all ecosystem members is also paramount. In this commentary, we—the Local Network Leads of the UK Reproducibility Network—outline our response to the UK House of Commons Science and Technology Committee’s inquiry on research integrity and reproducibility. We argue that coordinated change is needed to create (1) a positive research culture, (2) a unified stance on improving research quality, (3) common foundations for open and transparent research practice, and (4) the routinisation of this practice. For each of these areas, we outline the roles that individuals, institutions, funders, publishers, and Government can play in shaping the research ecosystem. Working together, these constituent members must also partner with sectoral and coordinating organisations to produce effective and long-lasting reforms that are fit-for-purpose and future-proof. These efforts will strengthen research quality and create research capable of generating far-reaching applications with a sustained impact on society

Reproducibility in management science

With the help of more than 700 reviewers we assess the reproducibility of nearly 500 articles published in the journal Management Science before and after the introduction of a new Data and Code Disclosure policy in 2019. When considering only articles for which data accessibility and hard- and software requirements were not an obstacle for reviewers, the results of more than 95% of articles under the new disclosure policy could be fully or largely computationally reproduced. However, for 29% of articles at least part of the dataset was not accessible to the reviewer. Considering all articles in our sample reduces the share of reproduced articles to 68%. These figures represent a significant increase compared to the period before the introduction of the disclosure policy, where only 12% of articles voluntarily provided replication materials, out of which 55% could be (largely) reproduced. Substantial heterogeneity in reproducibility rates across different fields is mainly driven by differences in dataset accessibility. Other reasons for unsuccessful reproduction attempts include missing code, unresolvable code errors, weak or missing documentation, but also soft- and hardware requirements and code complexity. Our findings highlight the importance of journal code and data disclosure policies, and suggest potential avenues for enhancing their effectiveness

UKRN Institutional tasks implied by the UK Open Research Data Concordat

This UKRN working paper outlines the tasks that are either explicitly or implicitly proposed for research institutions by the UK Open Research Data Concordat (2016). It also reflects on how those tasks are relevant to the recommendations from the 2018 report issued by the UK Government from the Open Research Data Task Force. The aim of this paper is to describe more clearly what UK research institutions might be expected to do to support open research data, which may be a helpful basis for sector-wide discussion on both those expectations and how they are being met. The tasks are grouped under the headings of organisational support, infrastructure, recognition, restrictions on open data, costs, and strategy

ORCC UKRN Primer on Open Access

This is an introductory guide for those working and considering working in the area of open access. It was drafted by members of the Open Research Competencies Coalition. Open Access (OA) refers to research that is published as digital, online, free of charge for reading, and free to re-use or share

ORCC UKRN Primer on Working in Open Research

This is an introductory guide for those working and considering working in the area of open research. It was drafted by members of the Open Research Competencies Coalition. There are many resources available on the topic of open research either aimed at those working in open research roles or more generally on open research practices. A list is included in the document

ORCC UKRN Primer on Working in Research Data Management

This is an introductory guide for those working and considering working in the area of research data management. It was drafted by members of the Open Research Competencies Coalition. Research Data Management (RDM) refers to the storage, access, and preservation of data produced by a given investigation. Data management practices cover the entire lifecycle of a project, from planning the investigation, conducting it, backing up data as it is created and used, to long term preservation of data deliverables after the research investigation has concluded

Reforms to improve reproducibility and quality must be coordinated across the research ecosystem: the view from the UKRN Local Network Leads

Many disciplines are facing a "reproducibility crisis", which has precipitated much discussion about how to improve research integrity, reproducibility, and transparency. A unified effort across all sectors, levels, and stages of the research ecosystem is needed to coordinate goals and reforms that focus on open and transparent research practices. Promoting a more positive incentive culture for all ecosystem members is also paramount. In this commentary, we-the Local Network Leads of the UK Reproducibility Network-outline our response to the UK House of Commons Science and Technology Committee's inquiry on research integrity and reproducibility. We argue that coordinated change is needed to create (1) a positive research culture, (2) a unified stance on improving research quality, (3) common foundations for open and transparent research practice, and (4) the routinisation of this practice. For each of these areas, we outline the roles that individuals, institutions, funders, publishers, and Government can play in shaping the research ecosystem. Working together, these constituent members must also partner with sectoral and coordinating organisations to produce effective and long-lasting reforms that are fit-for-purpose and future-proof. These efforts will strengthen research quality and create research capable of generating far-reaching applications with a sustained impact on society