Bioconductor: open software development for computational biology and bioinformatics.

The Bioconductor project is an initiative for the collaborative creation of extensible software for computational biology and bioinformatics. The goals of the project include: fostering collaborative development and widespread use of innovative software, reducing barriers to entry into interdisciplinary scientific research, and promoting the achievement of remote reproducibility of research results. We describe details of our aims and methods, identify current challenges, compare Bioconductor to other open bioinformatics projects, and provide working examples

Leveraging Semantics to Improve Reproducibility in Scientific Workflows

Reproducibility of published results is a cornerstone in scientific publishing and progress. Therefore, the scientific community has been encouraging authors and editors to publish their contributions in a verifiable and understandable way. Efforts such as the Reproducibility Initiative [1], or the Reproducibility Projects on Biology [2] and Psychology [3] domains, have been defining standards and patterns to assess whether an experimental result is reproducible

Creating a focus on Rigor and Reproducibility in a Health Sciences Library

OBJECTIVE For the Woodruff Health Sciences Center Library to determine if creating a focus on Open Science or Rigor and Reproducibility would be possible and accepted by researchers at Emory University. METHODS A working group of three librarians was created to determine if there was a need for the library to create a focus/culture of Open Science or Rigor and Reproducibility on campus. The working group interviewed 4 faculty/admin members including: the Deputy Chief Compliance Officer, an Associate Professor of Pharmacology, who teaches a course on Rigorous Research Methods, the Director of the CTSA’s Medical Ethics Section, and the Director of the Woodruff Health Sciences Center Data Science Initiative, to determine the need for the library to be a resource for Open Science or Rigor and Reproducibility. RESULTS The working group determined that creating a focus on Rigor and Reproducibility was a more important focus for the Library than Open Science. The group decided that in order for the Library to be seen as a resource for Rigor and Reproducibility, the library must employ the principles of Rigor and Reproducibility to its own research. The group created a training plan for the librarians in the Library in which the initial step was for all Librarians to attend a training session from the Center of Open Science entitled “Openness and Reproducibility Research Practices”. The group also determined the Library should hold at least one public outreach event a year based on the theme of Rigor and Reproducibility. The group worked with the Director of the Data Science Initiative to create a virtual seminar series based around Rigor and Reproducibility. CONCLUSIONS The Health Sciences Library deciding to create a focus on Rigor and Reproducibility has been an effective way to engage with faculty and researchers across campus to begin to create conversations around Rigor and Reproducibility and research on campus

Neurodata Without Borders: Creating a Common Data Format for Neurophysiology

The Neurodata Without Borders (NWB) initiative promotes data standardization in neuroscience to increase research reproducibility and opportunities. In the first NWB pilot project, neurophysiologists and software developers produced a common data format for recordings and metadata of cellular electrophysiology and optical imaging experiments. The format specification, application programming interfaces, and sample datasets have been released

Improving reproducibility of geospatial conference papers: lessons learned from a first implementation of reproducibility reviews

Ponència presentada a The 15th Munin Conference on Scholarly Publishing celebrat a Tromsø, Noruega, el 18 de novembre de 2020In an attempt to increase the reproducibility of contributions to a long-running and established geospatial conference series, the 23rd AGILE Conference on Geographic Information Science 2020 (https://agile-online.org/conference-2020) for the first time provided guidelines on preparing reproducible papers (Nüst et al., 2020) and appointed a reproducibility committee to evaluate computational workflows of accepted papers ( https://www.agile-giscience-series.net/review_process.html). Here, the committee’s members report on the lessons learned from reviewing 23 accepted full papers and outline future plans for the conference series. In summary, six submissions were partially reproduced by reproducibility reviewers, whose reports are published openly on OSF ( https://osf.io/6k5fh/). These papers are promoted with badges on the proceedings’ website (https://agile-giss.copernicus.org/articles/1/index.html). Compared to previous years’ submissions (cf. Nüst et al. 2018), the guidelines and increased community awareness markedly improved reproducibility. However, the reproduction attempts also revealed problems, most importantly insufficient documentation. This was partly mitigated by the non-blind reproducibility review, conducted after paper acceptance, where interaction between reviewers and authors can provide the input and attention needed to increase reproducibility. However, the reviews also showed that anonymisation and public repositories, when properly documented, can enable a successful reproduction without interaction, as was the case with one manuscript. Individual and organisational challenges due to the COVID-19 pandemic and the conference’s eventual cancellation increased the teething problems. Nevertheless, also under normal circumstances, future iterations will have to reduce the reviewer’s efforts to be sustainable, ideally by more readily executable workflows and a larger reproducibility committee. Furthermore, we discuss changes to the reproducibility review process and their challenges. Reproducibility reports could be made available to “regular” reviewers, or the reports could be considered equally for acceptance/rejection decisions. Insufficient information or invalid arguments for not disclosing material could then lead to a submission being rejected or not being sent out to peer review. Further organisational improvements are a publication of reviewers’ activities in public databases, making the guidelines mandatory, and collecting data on used tools/repositories, spent efforts, and communications. Finally, we summarise the revision of the guidelines, including their new section for reproducibility reviewers, and the status of the initiative “Reproducible Publications at AGILE Conferences” (https://reproducible-agile.github.io/initiative/), which we connect to related undertakings such as CODECHECK (Eglen et al., 2019). The AGILE Conference’s experiences may help other communities to transition towards more open and reproducible research publications

Advancing Science with VGI: Reproducibility and Replicability of Recent Studies using VGI

In scientific research, reproducibility and replicability are requirements to ensure the advancement of our body of knowledge. T his holds true also for VGI - related research and studies. However, the characteristics of VGI suggest particular difficulties in ensuring reproducibility and replicability . In this paper, we aim to examine the current situation in VGI - related research , and identify strategies to ensure realization of its full potential. To do so, we first investigate the different aspects of reprod ucibility and replicability and their impact on VGI - related research . These impacts are different depending on the objectives of the study. Therefore , we examine the study focus of VGI - related research to assess the current body of research and structure o ur assessment . Th is work is based on a rigorous review of the elements of reproducibility and a systematic mapping and analysis of 58 papers on the use of VGI in the crisis management field. Results of our investigation show that reproducibility issues related to data are a serious concern , while reproducibility issues related to analysis methods and processes face fewer challenges. Howe ver, since most studies still focus on analyzing the source data, reproducibility and replicability are still an unsolved problem in VGI - related research. Therefore, we show initiative s tackling the problem, and finally formulate strategies to improve the situatio

Open Research Pilot case studies: research integrity and reproducibility

This is the third blog in our series marking the end of the Open Research Pilot (a two-year initiative involving University of Cambridge research groups, University Research Support, and Wellcome Trust’s Open Research Team). Professor David Savage tells the Research Support Team how he is concerned with how to improve and ensure research integrity and reproducibility