TIER2 D3.1 - State-of-play on methods, tools, practices to increase reproducibility across diverse epistemic contexts

Abstract

This project deliverable has been submitted to, but not yet reviewed by the Research Executive Agency, and might thus be subject to change. TIER2, over the course of its three-year duration (2023-2025), aims to contribute to improving this situation in various ways. Key to our approach is to centre “epistemic diversity” (defined below) by selecting three broad research areas — social, life, and computer sciences, and two cross- disciplinary stakeholder groups of research publishers and funders — to systematically investigate the roles, nature, and meanings of reproducibility across contexts. Through coordinated co- creation with these communities, TIER2 aims to boost knowledge on reproducibility, create tools, engage communities, implement interventions and policy across different contexts to increase reproducibility where it is relevant. This Deliverable details work to provide the theoretical, evidential and strategic framework for the project. The aim is to capture the complexity in the meaning(s) of reproducibility across contexts, provide a conceptual framework that systematically relates epistemic diversity to reproducibility by identifying key research characteristics affecting the relevance and feasibility of different types of reproducibility, establish current levels of knowledge on which interventions work in which contexts (including in two specific cross-cutting research methods (qualitative and Machine Learning-driven research), and devise a strategic intervention logic for designing and implementing interventions that aim at sustainable behavioural change towards increased reproducibility. This work has been addressed through seven ambitious individual studies: • “Definitions of reproducibility” (Section 2) • “Epistemic diversity and Knowledge Production Modes” (Sec. 3) • “Scoping review and evidence mapping of interventions aimed at improving reproducible and replicable science” (Sec. 4) • “Review of conceptions and facilitators of and barriers to reproducibility of qualitative research” (Sec. 5) • “Review of conceptions and practices regarding reproducibility in Machine Learning (ML)- driven research” (Sec. 6) • “Changing behaviour in the academy: A strategy for improving research culture and practice” (Sec. 7) This work hence fills knowledge gaps to enable the mapping of “impact pathways”, i.e., the possible paths that connect input to output, outcome and impact (including linkages of causal mechanisms and drivers/barriers), to elucidate the routes to increased reproducibility across diverse contexts. This work is crucial to inform the future stages of TIER2, especially to design, implement and test a series of new tools and instruments (the “pilots”) conducted within TIER2 Work Packages 4 and 5