202,942 research outputs found
Mapping the Bid Behavior of Conference Referees
The peer-review process, in its present form, has been repeatedly criticized.
Of the many critiques ranging from publication delays to referee bias, this
paper will focus specifically on the issue of how submitted manuscripts are
distributed to qualified referees. Unqualified referees, without the proper
knowledge of a manuscript's domain, may reject a perfectly valid study or
potentially more damaging, unknowingly accept a faulty or fraudulent result. In
this paper, referee competence is analyzed with respect to referee bid data
collected from the 2005 Joint Conference on Digital Libraries (JCDL). The
analysis of the referee bid behavior provides a validation of the intuition
that referees are bidding on conference submissions with regards to the subject
domain of the submission. Unfortunately, this relationship is not strong and
therefore suggests that there exists other factors beyond subject domain that
may be influencing referees to bid for particular submissions
Favouritism and financial incentives: A natural experiment
Principals who exercise favouritism towards certain agents may harm those who are not so favoured. Other papers have produced evidence consistent with the presence of such favouritism but have been unable to consider methods for controlling it. We address this issue in the context of a natural experiment from English soccer, where one particular league introduced professional referees in 2001-02, thereby changing the financial incentives and monitoring regime faced by these referees. Because the change was not effected in all leagues, the ‘experiment’ has both cross-sectional and intertemporal dimensions. We study the effects of professional referees on an established measure of referee bias: length of injury time in close matches. We find that referees exercised favouritism prior to professionalism but not afterwards, having controlled for selection and soccer-wide effects. The results are consistent with a financial incentive effect as a result of professional referees and indicate that subtle aspects of principal-agent relationships (such as favouritism) are amenable to contractual influence.Favouritism, financial incentives, soccer, referee
Peer-review in a world with rational scientists: Toward selection of the average
One of the virtues of peer review is that it provides a self-regulating
selection mechanism for scientific work, papers and projects. Peer review as a
selection mechanism is hard to evaluate in terms of its efficiency. Serious
efforts to understand its strengths and weaknesses have not yet lead to clear
answers. In theory peer review works if the involved parties (editors and
referees) conform to a set of requirements, such as love for high quality
science, objectiveness, and absence of biases, nepotism, friend and clique
networks, selfishness, etc. If these requirements are violated, what is the
effect on the selection of high quality work? We study this question with a
simple agent based model. In particular we are interested in the effects of
rational referees, who might not have any incentive to see high quality work
other than their own published or promoted. We find that a small fraction of
incorrect (selfish or rational) referees can drastically reduce the quality of
the published (accepted) scientific standard. We quantify the fraction for
which peer review will no longer select better than pure chance. Decline of
quality of accepted scientific work is shown as a function of the fraction of
rational and unqualified referees. We show how a simple quality-increasing
policy of e.g. a journal can lead to a loss in overall scientific quality, and
how mutual support-networks of authors and referees deteriorate the system.Comment: 5 pages 4 figure
Asymptotics of the two-stage spatial sign correlation
Acknowledgments This research was supported in part by the Collaborative Research Grant 823 of the German Research Foundation. The authors wish to thank the editors and referees for their careful handling of the manuscript. They further acknowledge the anonymous referees of the article Spatial sign correlation (J. Multivariate Anal. 135, pages 89–105, 2015), who independently of each other suggested to further explore the properties of two-stage spatial sign correlation.Non peer reviewedPreprin
An Auction Market for Journal Articles
Economic articles are published very slowly. We believe this results mainly from the poor incentives referees face. We recommend that an auction market replace the current system for submitting papers and demonstrate a strict Pareto-improvement of equilibrium. Besides the benefits of speed, this mechanism increases the average quality of articles and journals and rewards editors and referees for their effort. In addition, the "academic dollars" for papers sold at auction go to the authors, editors and referees of cited articles. This income indicates academic productivity (facilitating decisions on tenure and promotion); its recirculation to journals further stimulates quality competition.Academic Journals;Academic Productivity;Market Design. JEL codes
- …