Reading Peer Review

This Element presents the background contexts and histories of peer review, the data-handling sensitivities of this type of research, the typical properties of reports in the journal to which the authors had access, a taxonomy of the reports, and their sentiment arcs. This title is also available as Open Access on Cambridge Core

University of South Alabama College of Medicine Annual Report for 2019-2020

The 2019-20 annual report for the University of South Alabama College of Medicine catalogues the accomplishments of our faculty, staff and students relating to teaching, research, other scholarship and community service.https://jagworks.southalabama.edu/com_report/1004/thumbnail.jp

Sharing Tacit Knowledge of Academic Publishing: How to Respond to Reviewer Comments

Abstract: Navigating academic publishing is trying and complicated for both junior and established scholars. Some of the demands from the process come from the hidden curriculum within academic culture which is exacerbated by the tacit knowledge that only some folks possess. In this editorial, I explain some of my personal struggles with academic publishing as a First in the Family (FiF)/First Generation Student (FGS). I utilize my understanding as a way to frame an issue within academic publishing that I have personally experienced and observed in my CJNSE editorship: responding to reviewer comments. I then outline the components academic authors must include when replying to their peers’ assessments of their manuscripts, and provide an example of a method of how to do so. Fittingly, the work of the authors, Review Mentors (graduate student peer mentors), and Senior Review Editors (PhDs with publishing experience) in this issue highlights the mentoring qualities of CJNSE and how tacit knowledge can be shared in this publishing environment. The topics discussed in this issue include the shortcomings within English language learning courses for immigrants and refugees to Canada (Lam); the different qualities and emotional intelligence required of department chairs (Cowley); the need for curricula change to include death, dying, and grief in elementary curricula (Durant); querying the principalship as a democratic process (Kendrick); questioning the idea of universal values for education policy educators (Hankey); the tensions between Deweyan and Confucian educational philosophies in English language learning in China (Peng); the factors that mediate a language teacher's corrective feedback decision (Chen); the factors that affect an individual’s likelihood of reporting sexual assault on a post-secondary campus (MacKenzie); the ways in which washback and curriculum agreement are interconnected methods of classroom instruction (Sultana); the limitations and utility of Kimberly Maich & Carmen Hall’s “Autism Spectrum Disorder in the Ontario Context: An Introduction,” and explaining the importance of Jen Gilbert’s “Sexuality in School: The Limits of Education” to educators, particularly those in Canada (Virani-Murji)

A multi-disciplinary perspective on emergent and future innovations in peer review

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments

A multi-disciplinary perspective on emergent and future innovations in peer review

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments

An environment for sustainable research software in Germany and beyond: current state, open challenges, and call for action

Research software has become a central asset in academic research. It optimizes existing and enables new research methods, implements and embeds research knowledge, and constitutes an essential research product in itself. Research software must be sustainable in order to understand, replicate, reproduce, and build upon existing research or conduct new research effectively. In other words, software must be available, discoverable, usable, and adaptable to new needs, both now and in the future. Research software therefore requires an environment that supports sustainability. Hence, a change is needed in the way research software development and maintenance are currently motivated, incentivized, funded, structurally and infrastructurally supported, and legally treated. Failing to do so will threaten the quality and validity of research. In this paper, we identify challenges for research software sustainability in Germany and beyond, in terms of motivation, selection, research software engineering personnel, funding, infrastructure, and legal aspects. Besides researchers, we specifically address political and academic decision-makers to increase awareness of the importance and needs of sustainable research software practices. In particular, we recommend strategies and measures to create an environment for sustainable research software, with the ultimate goal to ensure that software-driven research is valid, reproducible and sustainable, and that software is recognized as a first class citizen in research. This paper is the outcome of two workshops run in Germany in 2019, at deRSE19 - the first International Conference of Research Software Engineers in Germany - and a dedicated DFG-supported follow-up workshop in Berlin

University of South Alabama College of Medicine Annual Report for 2017-2018

This Annual Report of the College of Medicine catalogues accomplishments of our faculty, students, residents, fellows and staff in teaching, research, scholarly and community service during the 2017-2018 fiscal year.https://jagworks.southalabama.edu/com_report/1002/thumbnail.jp

Open Science in Software Engineering

Open science describes the movement of making any research artefact available to the public and includes, but is not limited to, open access, open data, and open source. While open science is becoming generally accepted as a norm in other scientific disciplines, in software engineering, we are still struggling in adapting open science to the particularities of our discipline, rendering progress in our scientific community cumbersome. In this chapter, we reflect upon the essentials in open science for software engineering including what open science is, why we should engage in it, and how we should do it. We particularly draw from our experiences made as conference chairs implementing open science initiatives and as researchers actively engaging in open science to critically discuss challenges and pitfalls, and to address more advanced topics such as how and under which conditions to share preprints, what infrastructure and licence model to cover, or how do it within the limitations of different reviewing models, such as double-blind reviewing. Our hope is to help establishing a common ground and to contribute to make open science a norm also in software engineering.Comment: Camera-Ready Version of a Chapter published in the book on Contemporary Empirical Methods in Software Engineering; fixed layout issue with side-note

University of South Alabama College of Medicine Annual Report for 2015-2016

This Annual Report of the College of Medicine catalogues recent accomplishments of our faculty, students, residents, fellows and staff in teaching, research, patient care, scholarly and community service activities during the 2015-16 academic year.https://jagworks.southalabama.edu/com_report/1000/thumbnail.jp

Double Trouble? The Communication Dimension of the Reproducibility Crisis in Experimental Psychology and Neuroscience

Most discussions of the reproducibility crisis focus on its epistemic aspect: the fact that the scientific community fails to follow some norms of scientific investigation, which leads to high rates of irreproducibility via a high rate of false positive findings. The purpose of this paper is to argue that there is a heretofore underappreciated and understudied dimension to the reproducibility crisis in experimental psychology and neuroscience that may prove to be at least as important as the epistemic dimension. This is the communication dimension. The link between communication and reproducibility is immediate: independent investigators would not be able to recreate an experiment whose design or implementation were inadequately described. I exploit evidence of a replicability and reproducibility crisis in computational science, as well as research into quality of reporting to support the claim that a widespread failure to adhere to reporting standards, especially the norm of descriptive completeness, is an important contributing factor in the current reproducibility crisis in experimental psychology and neuroscience