The removal of the Beall’s list of predatory publishers last month caused consternation and led to calls in some quarters for a new equivalent to be put in its place. Cameron Neylon explains why he has never been a supporter of the Beall’s list and outlines why he believes the concept of the blacklist itself is fundamentally flawed. Not only are blacklists incomplete by definition, they are highly susceptible to legal challenge and vulnerable to personal bias. Scholars should be able to decide for themselves what is a good venue from which to communicate their work

Neylon, Cameron

LSE Research Online

2/21/2017Blacklists are technically infeasible, practically unreliable andunethical. Period.blogs.lse.ac.uk/impactofsocialsciences/2017/02/21/blacklists-are-technically-infeasible-practically-unreliable-and-unethical-period/The removal of the Beall’s list of predatory publishers last month caused consternation and led tocalls in some quarters for a new equivalent to be put in its place. Cameron Neylon explains why hehas never been a supporter of the Beall’s list and outlines why he believes the concept of theblacklist itself is fundamentally flawed. Not only are blacklists incomplete by definition, they arehighly susceptible to legal challenge and vulnerable to personal bias. Scholars should be able todecide for themselves what is a  good venue from which to communicate their work.The last weekend of January was a bad one for poorly designed blacklists. But prior to this anotherblacklist was also the subject of significant discussion. Beall’s list of so-called ‘predatory’ journals and publishersvanished from the web in mid-January. There is still no explanation for why, but the most obvious candidate is thatlegal action, threatened or real, was the cause of it being removed. Since it disappeared many listservs and groupshave been asking what should be done. My answer is pretty simple. Absolutely nothing.It won’t surprise anyone that I’ve never been a supporter of the list. Early on I held the common view that Beall wasproviding a useful service, albeit one that overstated the problem. But as things progressed my concerns grew. Thecriticisms have been rehearsed many times so I won’t delve into the detail. Suffice to say Beall has a strongly anti-OA stance, was clearly partisan on specific issues, and antagonistic – often without being constructively critical – topublishers experimenting with new models of review. But most importantly his work didn’t meet minimum scholarlystandards of consistency and validation. Walt Crawford is the go-to source on this having done the painstaking workof actually documenting the state of many of the ‘publishers’ on the list but it seems like only a small percentage ofthe blacklisted publishers were ever properly documented by Beall.1/4Image credit: What’s on the blacklist? by opensource.com. This work is licensed under a CCBY-SA 2.0 license.Does that mean that it’s a good thing the lists are gone? That really depends on your view of the scale of theproblem. The usual test case of the limitations of free speech is whether it is OK to shout “FIRE!” in a crowdedtheatre when there is none. Depending on your perspective you might feel that our particular theatre has anythingfrom a candle onstage to a raging inferno under the stalls. From my perspective there is a seriousproblem, although the problem is not what most people worry about, and is certainly not limited to open accesspublishers. And the list didn’t help.But the real reason the list doesn’t help isn’t because of its motivations or its quality. It’s a fundamental structuralproblem with blacklists. They don’t work, they can’t work, and they never will work. Even when put together by ‘thegood guys’ they are politically motivated. They have to be because they’re also technically impossible to make work.Blacklists are technically infeasibleBlacklists are never complete. Listing is an action that has to occur after any given agent has acted in a way thatmerits listing. Whether that listing involves being called before the House Committee on Un-American Activities orbeing added to an online list it can only happen after the fact. Even if it seems to happen before the fact, that justmeans that the real criteria are a lie. The listing still happens after the real criteria were met, whether that is being aJewish screenwriter or starting up a well-intentioned but inexpert journal in India.Whitelists by contrast are by definition always complete. They are a list of all those agents that have been certifiedas meeting a certain level of quality assurance. There may be many agents that could meet the requirements, but ifthey are not on the list they have not yet been certified, because that is the definition of the certification. That mayseem circular but the logic is important. Whitelists are complete by definition. Blacklists are incomplete by definition.And that’s before we get to the issue of criteria to be met vs criteria to be failed.Blacklists are practically unreliableA lot of people have been saying “we need a replacement for the list because we were relying on it”. This, to beblunt, was stupid. Blacklists are discriminatory in a way that makes them highly susceptible to legal challenge. All2/4that is required is that it be shown that either the criteria for inclusion are discriminatory (or libellous) or that they arebeing applied in a discriminatory fashion. The redress is likely to be destruction of the whole list. Again, by contrastwith a whitelist the redress for discrimination is inclusion. Any litigant will want to ensure that the list is maintained sothey get listed. Blacklists are at high risk of legal takedown and should never be relied upon as part of a broadersystem. Use a whitelist or whitelists (and always provide a mechanism for showing that something that isn’t yetcertified should still be included in the broader system).If your research evaluation system relies on a blacklist it is fragile, as well as likely being discriminatory.Blacklists are inherently unethicalBlacklists are designed to create and enforce collective guilt. Because they use negative criteria they will necessarilyinclude agents that should never have been caught up. Blacklisting entire countries means that legal permanentresidents, indeed, it seems, even airline staff, were unable to board flights to the US last month. Blacklistingpublishers seeking to experiment with new forms of review or new business models both stifles innovation anddiscriminates against new entrants. Calling out bad practice is different. Pointing to one organisation and saying itsbusiness practices are dodgy is perfectly legitimate if done transparently, ethically and with due attention toevidence. Collectively blaming a whole list is not.Quality assurance is hard work and doing it transparently, consistently and ethically is even harder. Consigning anorganisation to the darkness based on a misstep or, worse, a failure to align with a personal bias, is actually quiteeasy, hard to audit effectively and usually oversimplifying a complex situation. To give a concrete example, DOAJmaintains a list of publishers that claim to have DOAJ certification but which do not. Here the ethics are clear, theDOAJ is a whitelist that is publicly available in a transparent form (whether or not you agree with the criteria).Publishers that claim membership they don’t have can be legitimately, and individually, called out. Such behaviour iscause for serious concern and appropriate to note. But DOAJ does not then propose that these journals should becast into outer darkness, it merely notes the infraction.So what should we do? Absolutely nothing!We already have plenty of perfectly good whitelists. PubMed listing, Web of Science listing, Scopus listing, DOAJlisting. If you need to check whether a journal is running traditional peer review at an adequate level, use somecombination of these according to your needs. Also ensure there is a mechanism for making a case for exceptions,3/4but use whitelists not blacklists by default.Authors should check with services like ThinkCheckSubmit or Quality Open Access Market  if they want data to helpthem decide whether a journal or publisher is legitimate. But above all scholars should be capable of making thatdecision for themselves. If we aren’t able to make good decisions on the venue to communicate our work then wedo not deserve the label ‘scholar’.Finally, if you want a source of data on the scale of the problem of dodgy business practices in scholarly publishingthen delve into Walt Crawford’s meticulous, quantitative, comprehensive and above all documented work on thesubject. It is by far the best data on both the number of publishers with questionable practices and the number ofarticles being published. If you’re serious about looking at the problem then start there.But we don’t need another f###ing list.This post originally appeared on Cameron Neylon’s personal blog, Science in the Open, and is reposted under aCC0 1.0 license.Note: This article gives the views of the author, and not the position of the LSE Impact Blog, nor of the LondonSchool of Economics. Please review our comments policy if you have any concerns on posting a comment below.About the authorCameron Neylon is Professor of Research Communications and the Centre for Culture and Technology, CurtinUniversity, and a well-known agitator for opening up the process of research. He speaks regularly on issues of OpenScience including Open Access publication, Open Data, and open source as well as the wider technical andsocial issues of applying the opportunities the internet brings to the practice of science. He was named as a SPARCInnovator in July 2010 for work on the Panton Principles,was a co-author of the Altmetrics manifesto and is a proudrecipient of the Blue Obelisk for contributions to open data. He writes regularly at his blog, Science in the Open.Copyright © The Author (or The Authors) - Unless otherwise stated, this work is licensed under a CreativeCommons Attribution Unported 3.0 License.4/4

Blacklists are technically infeasible, practically unreliable and unethical. Period.

Abstract

Similar works

Full text

Available Versions

LSE Research Online