The Academic Review Process: How Can We Make it More Efficient?

Recently many editors try to reduce the turnaround times of academic journals. Shorter turnaround times, however, will induce many additional submissions of low-quality papers, increasing significantly the workload of editors and referees, and the number of rejections prior to publication. I suggest several ideas how editors can shorten turnaround times and four ideas how they can still avoid frivolous submissions, thus improving the review process efficiency: higher submission fees; requiring authors to review papers in proportion to their submissions; using differential editorial delay – letting low-quality papers wait more; and banning papers from being submitted after a certain number of rejections.Academic publishing, first response times, editorial process, review process, refereeing

Peer-review in a world with rational scientists: Toward selection of the average

One of the virtues of peer review is that it provides a self-regulating selection mechanism for scientific work, papers and projects. Peer review as a selection mechanism is hard to evaluate in terms of its efficiency. Serious efforts to understand its strengths and weaknesses have not yet lead to clear answers. In theory peer review works if the involved parties (editors and referees) conform to a set of requirements, such as love for high quality science, objectiveness, and absence of biases, nepotism, friend and clique networks, selfishness, etc. If these requirements are violated, what is the effect on the selection of high quality work? We study this question with a simple agent based model. In particular we are interested in the effects of rational referees, who might not have any incentive to see high quality work other than their own published or promoted. We find that a small fraction of incorrect (selfish or rational) referees can drastically reduce the quality of the published (accepted) scientific standard. We quantify the fraction for which peer review will no longer select better than pure chance. Decline of quality of accepted scientific work is shown as a function of the fraction of rational and unqualified referees. We show how a simple quality-increasing policy of e.g. a journal can lead to a loss in overall scientific quality, and how mutual support-networks of authors and referees deteriorate the system.Comment: 5 pages 4 figure

Concentration in Knowledge Output:A Case of Economics Journals

Journals moderate knowledge activity in economics. The activity of publishing article in professional journal forms significant part of knowledge output. Output of economics articles has been growing over the time. We examine an important question: Is there any case of institutional or location concentration in knowledge production? This paper analyses concentration indicators specific to economics journals and explores link between publication process and concentration. The analysis of various concentration measures present evidence for institutional-geographic-area-author concentration in Knowledge production in Economics. High concentration levels indicate possibility of institutional lock-in. The literature provides evidence for myopic refereeing, editorial favouritism and the presence of ‘lock-in’ effect. The achievement in journal publication is influenced by factors like institutional affiliation, propitious circumstances etc. Discussion carried out in this paper hints the possibility of causal link between unfair process and unfair outcome.Knowledge,Lotka's Law,Fourier Series

A Novel Solution to Academic Publishing

Scientists have complained about the inconsistency and politics of academic publishing for hundreds of years. Among the explanations offered are that evaluators lack time and use shortcuts, that they lack the expertise to judge things properly, that they can't put aside personal biases and we must hide the names of authors, and that they are conscientious instead of creative and cannot judge new ideas. All of these are actually wrong. As a literary analyst, I spent the last ten years independently studying this same problem in book and movie production. I've found that the human decision-making apparatus doesn't work the way we think, and the solutions based on this misunderstanding could never have solved the problem. In this paper, we present the first method that actually can, which is a technique adapted from computer hacking, as well as a new view of how our brains make choices

Exogenous Treatment and Endogenous Factors: Vanishing of Omitted Variable Bias on the Interaction Term

Whether interested in the differential impact of a particular factor in various institutional settings or in the heterogeneous effect of policy or random experiment, the empirical researcher confronts a problem if the factor of interest is correlated with an omitted variable. This paper presents the circumstances under which it is possible to arrive at a consistent estimate of the mentioned effect. We find that if the source of heterogeneity and omitted variable are jointly independent of policy or treatment, then the OLS estimate on the interaction term between the treatment and endogenous factor turns out to be consistent.treatment effect; heterogeneity; policy evaluation; random experiments; omitted variable bias

Discrimination and nepotism: the efficiency of the anonymity rule.

The paper considers two categories of discrimination: 'discrimination against' and 'discrimination in favor', which Becker coins 'nepotism'. The paper develops an experimental test to distinguish between these two types of discrimination. The experiment compares the behavior towards individuals of different groups with the behavior towards anonymous individuals (those having no clear group affiliation). We illustrate the two attitudes by considering two segmented societies: Belgian society, with its linguistic segmentation between the Flemish and the Walloons, and Israeli society, where we focus on religious versus secular segmentation. In Belgium, we find evidence of discrimination against. Both the Walloons and the Flemish treat people of their own group in the same way as anonymous individuals while discriminating against individuals of the other group. In contrast, the behavior of ultra-orthodox religious Jews in Israel can be categorized as nepotism: they favor members of their own group while treating anonymous individuals in the same way as secular individuals. The distinction between the different types of discrimination is important in evaluating the effectiveness and the efficiency consequences of anti-discriminatory legislations.Discrimination; Efficiency; Effectiveness; Legislation;

Peer Review

Help understanding peer review for successful academic research

GENDER'S ROLE IN MANUSCRIPT ACCEPTANCE: SEX IN THE JOURNAL

Women authors fare poorly at the hands of referees in some economics journals, especially when the review process is not blind. Using data on the 155 manuscripts submitted to the NJARE for publication during the period 1984-88, we found no evidence of differential referee acceptance rates for manuscripts with female and male lead authors.Teaching/Communication/Extension/Profession,

Open Science in Software Engineering

Open science describes the movement of making any research artefact available to the public and includes, but is not limited to, open access, open data, and open source. While open science is becoming generally accepted as a norm in other scientific disciplines, in software engineering, we are still struggling in adapting open science to the particularities of our discipline, rendering progress in our scientific community cumbersome. In this chapter, we reflect upon the essentials in open science for software engineering including what open science is, why we should engage in it, and how we should do it. We particularly draw from our experiences made as conference chairs implementing open science initiatives and as researchers actively engaging in open science to critically discuss challenges and pitfalls, and to address more advanced topics such as how and under which conditions to share preprints, what infrastructure and licence model to cover, or how do it within the limitations of different reviewing models, such as double-blind reviewing. Our hope is to help establishing a common ground and to contribute to make open science a norm also in software engineering.Comment: Camera-Ready Version of a Chapter published in the book on Contemporary Empirical Methods in Software Engineering; fixed layout issue with side-note