Improving research quality: the view from the UK Reproducibility Network institutional leads for research improvement

The adoption and incentivisation of open and transparent research practices is critical in addressing issues around research reproducibility and research integrity. These practices will require training and funding. Individuals need to be incentivised to adopt open and transparent research practices (e.g., added as desirable criteria in hiring, probation, and promotion decisions, recognition that funded research should be conducted openly and transparently, the importance of publishers mandating the publication of research workflows and appropriately curated data associated with each research output). Similarly, institutions need to be incentivised to encourage the adoption of open and transparent practices by researchers. Research quality should be prioritised over research quantity. As research transparency will look different for different disciplines, there can be no one-size-fits-all approach. An outward looking and joined up UK research strategy is needed that places openness and transparency at the heart of research activity. This should involve key stakeholders (institutions, research organisations, funders, publishers, and Government) and crucially should be focused on action. Failure to do this will have negative consequences not just for UK research, but also for our ability to innovate and subsequently commercialise UK-led discovery

Fairness and Transparency in Crowdsourcing

International audienceDespite the success of crowdsourcing, the question of ethics has not yet been addressed in its entirety. Existing efforts have studied fairness in worker compensation and in helping requesters detect malevolent workers. In this paper, we propose fairness axioms that generalize existing work and pave the way to studying fairness for task assignment, task completion, and worker compensation. Transparency on the other hand, has been addressed with the development of plug-ins and forums to track workers' performance and rate requesters. Similarly to fairness, we define transparency axioms and advocate the need to address it in a holistic manner by providing declarative specifications. We also discuss how fairness and transparency could be enforced and evaluated in a crowdsourcing platform

Benefits of the Snakemake Workflow Management Software in Comparison to Traditional Programming (Paper)

Tools surrounding bioinformatics have increased data acquisition and accuracy significantly, especially with near-real time results using nanopore DNA sequencing. With large amounts of data, reproducibility is of high importance, and long workflows can become convoluted. Snakemake, built on the Common Workflow Language and Python, aims to alleviate this with readable formatting, reproducibility, and portability for any machine. Using 97 fastq files, the usability of these three traits were compared between a Bash and Snakemake workflow using a range of one to twelve threads. In every test, Snakemake was faster than Bash. At its fastest, Snakemake was 27% faster than Bash. Reproducibility of both workflows was verified using an MD5 hash of results. The hashes differed between the workflows; this may be a result of executing the workflows in two different terminal environments. Despite this, it is a valid method of validating reproducibility between tests within individual workflows. Outside speed tests, Snakemake offers quality of life features that allow it to pull ahead from Bash. Containerization of workflows using Conda is one example of this. The ability to require specific versions of software within a workflow boosts reproducibility. Additionally, portability is increased because the container can be deployed almost anywhere, and the required software can be downloaded on an as-needed basis. With readability comes maintainability. Snakemake will almost always pull ahead of Bash in this regard with its simple input, output, and shell fields. The field of Bioinformatics is moving very quickly, and it can be difficult for traditional Bash scripts to keep up in certain aspects. While Bash is paramount in the execution of some software, more powerful tools like Snakemake are required to handle the execution of an entire, complex workflow.https://openprairie.sdstate.edu/honors_isp/1007/thumbnail.jp

Justify your alpha

Benjamin et al. proposed changing the conventional “statistical significance” threshold (i.e.,the alpha level) from p ≤ .05 to p ≤ .005 for all novel claims with relatively low prior odds. They provided two arguments for why lowering the significance threshold would “immediately improve the reproducibility of scientific research.” First, a p-value near .05provides weak evidence for the alternative hypothesis. Second, under certain assumptions, an alpha of .05 leads to high false positive report probabilities (FPRP2 ; the probability that a significant finding is a false positive

Coordination in Distributed Agile Software Development: Insights from a COTS-based Case Study

This study investigates the practices of a development team that uses an Agile system of working where some team members and stakeholders were distributed geographically and temporally. The focus of the investigation was to study the dependencies and related coordination activities as the team collaborated on their work, which was the installation and customization of a complex Commercial Off-The-Shelf (COTS) software system in this case. We collected data by interviewing eight key team members and observing three team meetings over a 2-month period. We made detailed field notes and used thematic analysis to identify the key globally distributed dependencies in the development process. We identify and discuss the coordination mechanisms and tools that address these dependencies, along with the main coordination challenges. We conclude by discussing some ideas and lessons learned by the participants which we expect to be useful for other teams in a similar context

The Japanese Clinical Practice Guidelines for Management of Sepsis and Septic Shock 2016 (J-SSCG 2016)

Background and purposeThe Japanese Clinical Practice Guidelines for Management of Sepsis and Septic Shock 2016 (J-SSCG 2016), a Japanese-specific set of clinical practice guidelines for sepsis and septic shock created jointly by the Japanese Society of Intensive Care Medicine and the Japanese Association for Acute Medicine, was first released in February 2017 and published in the Journal of JSICM, [2017; Volume 24 (supplement 2)] https://doi.org/10.3918/jsicm.24S0001 and Journal of Japanese Association for Acute Medicine [2017; Volume 28, (supplement 1)] http://onlinelibrary.wiley.com/doi/10.1002/jja2.2017.28.issue-S1/issuetoc.This abridged English edition of the J-SSCG 2016 was produced with permission from the Japanese Association of Acute Medicine and the Japanese Society for Intensive Care Medicine.MethodsMembers of the Japanese Society of Intensive Care Medicine and the Japanese Association for Acute Medicine were selected and organized into 19 committee members and 52 working group members. The guidelines were prepared in accordance with the Medical Information Network Distribution Service (Minds) creation procedures. The Academic Guidelines Promotion Team was organized to oversee and provide academic support to the respective activities allocated to each Guideline Creation Team. To improve quality assurance and workflow transparency, a mutual peer review system was established, and discussions within each team were open to the public. Public comments were collected once after the initial formulation of a clinical question (CQ) and twice during the review of the final draft. Recommendations were determined to have been adopted after obtaining support from a two-thirds (> 66.6%) majority vote of each of the 19 committee members.ResultsA total of 87 CQs were selected among 19 clinical areas, including pediatric topics and several other important areas not covered in the first edition of the Japanese guidelines (J-SSCG 2012). The approval rate obtained through committee voting, in addition to ratings of the strengths of the recommendation, and its supporting evidence were also added to each recommendation statement. We conducted meta-analyses for 29 CQs. Thirty-seven CQs contained recommendations in the form of an expert consensus due to insufficient evidence. No recommendations were provided for five CQs.ConclusionsBased on the evidence gathered, we were able to formulate Japanese-specific clinical practice guidelines that are tailored to the Japanese context in a highly transparent manner. These guidelines can easily be used not only by specialists, but also by non-specialists, general clinicians, nurses, pharmacists, clinical engineers, and other healthcare professionals

Visibility of Work: How Digitalization Changes the Workplace

Digitalization is rapidly reshaping our workplaces, as digital technologies often change individual’s work and collective work practices in significant but unpredictable ways. In this paper, we look at how digitalization changes the nature of work regarding work visibility, and examine this in the context of business-to-business (B2B) sales work. We report on a single case study using a practice approach to examine B2B sales work in a small SaaS company. Our findings show how the visibility of B2B sales work changes due to digitalization, increasing the visibility of work in relation to co-workers while decreasing it in relation to customers. Based on these findings, we discuss the complex tradeoffs in making work visible to different audiences and the gradual and constructed nature of work visibility