Collins, Harry

English

Villanova University School of Law: Digital Repository

Volume 67 Issue 5 2022 Villanova Law Review Norman J. Shachoy Symposium: The Consensus Rule: A New Approach To Admissibility Of Scientific Evidence Article 3 2-14-2023 The Owls: Some Difficulties in Judging Scientific Consensus Harry Collins Follow this and additional works at: https://digitalcommons.law.villanova.edu/vlr  Part of the Evidence Commons, Judges Commons, Jurisprudence Commons, and the Science and Technology Law Commons Recommended Citation Harry Collins, The Owls: Some Difficulties in Judging Scientific Consensus, 67 Vill. L. Rev. 877 (2023). Available at: https://digitalcommons.law.villanova.edu/vlr/vol67/iss5/3 This Symposia is brought to you for free and open access by the Journals at Villanova University Charles Widger School of Law Digital Repository. It has been accepted for inclusion in Villanova Law Review by an authorized editor of Villanova University Charles Widger School of Law Digital Repository. 2022]THE OWLS: SOME DIFFICULTIES IN JUDGINGSCIENTIFIC CONSENSUSHARRY COLLINS*EDWARD Cheng’s arguments for abandoning the Daubert rule are en-tirely convincing.  Sociologists think in terms of institutions as formedby their cultures: the law has one set of cultural expectations for the waydecisions are made, science has a different set.  Cheng has seen that whereexpert witnesses are concerned, scientific decisions are being made ac-cording to legal procedures, and it doesn’t work.  Another way of puttingit is that science and the law are defined, in part, by their “locus of legiti-mate interpretation” (LLI)—who is legitimately entitled to interpret whatis going on in that cultural domain, the answer establishing the meaningof the knowledge-making culture.  In science, the LLI is restricted to theproducers of science, or people close to the producers; that community,for example, is where the peer reviewers are drawn from.  Contrast thiswith the frontiers of fine art, where what counts is defined more by galleryowners and newspaper critics—people close to the consumers, not theproducers.In jury trials the LLI extends to the ordinary citizen.  Cheng is describingwhat goes wrong when such an LLI is imposed on science.1Cheng’s solution is to shift from trying to establish the scientific factsusing the procedures of the courtroom to establishing them using the in-stitutions of science, and merely reporting the outcome to the court—“thisis the scientific consensus on this matter.”  We are always asking Plato’squestion, “who guards the guardians?” and Cheng’s solution is a move in* Distinguished Research Professor, School of Social Sciences, CardiffUniversity.  Elected a Fellow of the British Academy in 2012.1. In the early days of the cultural revolution in the understanding of thenature of science, it was refreshing to realize that forensic science should not bedone by an official body organized by the police but should have representativesfrom both defense and prosecution.  But now we are seeing that we must go tosomething more neutral because the adversarial style of the courtroom likewisecannot give rise to good science.(877)1Collins: The Owls: Some Difficulties in Judging Scientific ConsensusPublished by Villanova University Charles Widger School of Law Digital Repository, 2023878 VILLANOVA LAW REVIEW [Vol. 67: p. 877the right direction.  Refreshingly, it is a move away from the fashion, popu-lar in various academic and political circles, to proclaim that the ultimateguardians should not be technical elites but “the people,” taken to be thesource of a kind of magical collective wisdom even in technical matters.Legal juries usually work because their decisions are of the kind that ordi-nary citizens learn to make in their day-to-day lives, such as who can betrusted and who cannot—so no magic is involved, only “ubiquitous exper-tise.”  Ubiquitous expertise is the deep expertise of the citizen in knowinghow to speak their native language and live in their society—without ubiq-uitous expertise we’d all be strangers in our own country, and the typicaljury draws on those abilities.  But while scientific consensus is also formedthrough trust developed in face-to-face interaction, the expertise neededto take part in and understand the process is particular to the science inquestion and is developed among the specific core-group of scientists.2These core-groups are generally small and, where the science is difficult,they guard their boundaries jealously.  From long experience I can tell youthat one of the very hardest things you have to learn in becoming a mem-ber of a scientific core-set is learning how to trust and who to trust amongthose you encounter in the small group making the science.3For good or ill, today we have before our eyes about as convincing aset of demonstrations as we could have, for the potential for the citizen’subiquitous expertise to fail, when we come to complicated technical mat-ters such as vaccination.  It is disappointing that certain colleagues in thesociology of science are determined to blame the science, or the institu-tion of science, not the people, for this failure, sacrificing the chance tosuppress some of the chaos that is leading democracy into populism.4Cheng knows that the jury’s ubiquitous expertise is not going to be ade-quate when faced with deciding between competing scientific experts,even when supplemented by a judge who has mugged-up certain supposedrules and features of science developed when it was still believed that there2. The concept of ubiquitous expertise is a feature of the “Periodic Table ofExpertises” found in, for example, HARRY M. COLLINS & ROBERT EVANS, RETHINK-ING EXPERTISE 14 (2007).  The need for trust in the formation of scientific knowl-edge has long been understood under the “second wave” of social studies ofscience. See H.M. Collins & Robert Evans, The Third Wave of Science Studies: Studies ofExpertise and Experience, 32 SOC. STUDS. SCI. 235 (2002) (describing “three waves ofscience studies”).  For particular instances of the complexities of developing trustin contemporary science, see H.M. Collins, Tacit Knowledge, Trust, and the Q of Sap-phire, 31 SOC. STUDS. SCI. 71 (2001); HARRY M. COLLINS, GRAVITY’S KISS: THE DETEC-TION OF GRAVITATIONAL WAVES 322 (2017).3. A “core-set” is the small set of scientists from whom consensus eventuallyemerges; core-sets may be riven by disagreement.  “Core-groups” have excludedthe major disagreements.4. The “cultural turn” in the social study of science is easily absorbed into thechaos of beliefs that encourage the slogans underpinning populism and fascism.See Harry M. Collins, Robert Evans & Martin Weinel, STS as Science or Politics?, 47SOC. STUDS. SCI. 580 (2017).2Villanova Law Review, Vol. 67, Iss. 5 [2023], Art. 3https://digitalcommons.law.villanova.edu/vlr/vol67/iss5/32022] OWLS DIFFICULTY CONSENSUS 879was a “formula” for science involving corroboration, or falsification, orsome such philosophical touchstone.5Unfortunately, Plato’s problem won’t be made to go away simply byturning to scientific consensus unless that consensus is already solidlyformed.  Where there are disputes, the substance of scientific consensusand the strength of the consensus can be hard to establish, and in bothpolicy making and the courtroom, we need to know both substance andstrength.  Even though science is the best institution for reaching the bestkind of consensus on matters scientific, it is far from perfect and still fur-ther from perfect when it comes to outsiders understanding what that con-sensus is.  Let’s set out some of the reasons.First, scientists come in various types.  Many of them don’t under-stand how science works.  One of the most famous scientists of all timepointed this out.  Richard Feynman (is said to have) said, “The philosophyof science is as useful to scientists as ornithology is to birds.”  What hemeant is that scientists, like other skilled professionals, learn how to prac-tice their skills without having to think about them first, and he was right.But it means that scientists are not always the best people to reflect on thenature of their skills, since there is no reason for them to do so; and it alsomeans they are ripe subjects to absorb and retransmit all kinds of mytho-logical versions of what they do.  These myths are good for teaching sci-ence (and the textbooks are full of heroic stories about famousexperiments and “Eureka events”), and teaching science this way does noharm until we have to think about how to apply science to policy, or howto apply it in the courtroom.  Many scientists are powerful predators whosegaze is narrowly focused forward in pursuit of the next objective in theirdomain of truth.  Scientists of that sort think that talking in terms of con-sensus is to take the easy way out: the driving ethos is that something iseither true or it isn’t, and that view is an important component of scien-tific culture.6  We give the nickname “eagles” to such scientists.  Some-times, commitment to the truth, as they see it, can give rise to a tenacious“fringe science,” which no outsider could distinguish from the science it-self in terms of the technical proficiency of its champions, but which liesoutside the consensus and won’t go away.5. Scientific truth-making is different to legal truth-making: scientific truth-making is long-term and continually open to revision; legal truth-making is limitedby timetable and formal procedures.  Legal advocacy also permits technical tricksin support of a client, whereas scientific truth-making has truth, not defense of aposition, as its central value.6. I embarked on a rather depressing series of emails with such a scientistwhose views were not part of the consensus.  I asked, I paraphrase, “though youmay not believe ‘such and such,’ if asked, would you agree that it represents themainstream view and your claims do not?”  I could not get an answer outside ofrepeated explanations of why his views were right and the other views were wrong.But scientists could probably soon learn to understand what consensus means andlearn to recognize it in others even when it conflicted with their own views.3Collins: The Owls: Some Difficulties in Judging Scientific ConsensusPublished by Villanova University Charles Widger School of Law Digital Repository, 2023880 VILLANOVA LAW REVIEW [Vol. 67: p. 877Second, on the other hand, there are a few scientists, better thoughtof as “owls,” who are among the most brilliant understanders of the socialsubtleties of their profession:  their ideas have helped to form the under-pinnings of social studies of science.  While deeply immersed in their sci-entific craft, owls are still able to turn their heads through 180 degrees andreflect upon the sociological and philosophical setting which comprisesthe scientific “air” they fly in.  There is a complementary set of owls in thesocial studies of science, who start out by reflecting on the social “air” butwork to understand the technical perspective of the scientific fields theystudy, so they, too, can look both ways.7  A suitable selection of owls fromnatural and social science, given the topic under consideration, could, wehave suggested elsewhere, form a committee called “The Owls,” who coulddeliberate and give an answer to the substance and strength of consensusin domains of science to which their experience was a match.  That couldat least move us an asymptotic increment closer to resolving the quis cus-todiet problem where science and technology related policy is concerned.We have not had much take-up, but we still think there is something in it.The idea would, of course, be hard to apply within the timeframe of thecourtroom where a dispute is serious and ongoing.But we must not allow the perfect to drive out the good: moving inthe direction of consensus has to be better that what we have now.  Oneimmediate benefit of thinking in terms of consensus is that the language isaccessible, whereas any attempt to establish the scientific truth of the mat-ter in a debate among lay-persons is derailed by the opacity of technicallanguage—we are always going to be “blinded by science.”  But the lan-guage of consensus is the language of everyday.  For example, when wewanted a description for highly qualified scientists whose ideas were onceimportant but are no longer taken seriously, even though they continue tocleave to their technical view, we chose a term from the supermarket—they are scientists who are “past their sell-by date”; when we wanted adescription for highly qualified scientists corruptly inventing scientificdoubt, we talked of “fake scientific controversies.”  We cannot get awayfrom the fact that you need some scientific nous to know who is whom, butat least you know what you are looking for.Here are some more reasons why owls8 are needed for this kind oftask:(1) Not all the technological expertise that has a legitimate placein a technological decision comes from qualified scientists:there are experience-based experts whose views should betaken into account even if they don’t coincide with the views of7. The determined believers in the wisdom of the public in the domain ofscience and technology are the social science equivalent of eagles.8. The idea of “The Owls” is first discussed (and at greater length in respectof technological policymaking) in HARRY M. COLLINS & ROBERT EVANS, WHY DE-MOCRACIES NEED SCIENCE (2017).4Villanova Law Review, Vol. 67, Iss. 5 [2023], Art. 3https://digitalcommons.law.villanova.edu/vlr/vol67/iss5/32022] OWLS DIFFICULTY CONSENSUS 881the qualified experts.  Owls are better than scientist-eagles athandling this uncomfortable reality without taking the ideato the extreme view that only citizens know when the scien-tific emperor has no clothes.(2) On the other hand, some of the views of the best qualifiedscientists should not be taken into account in forming the con-sensus because, as we have said, there are well-establishedand well qualified bodies of “fringe” scientists publishing pa-pers in journals which are difficult to distinguish from main-stream journals unless you are an “insider.”9  An alien tryingto learn terrestrial science from the literature would get ithopelessly wrong.10  A grasp of both the science and the so-cial science of a domain are needed to sort out any particu-lar case.9. The Daubert criteria just would not apply here.  For some sense of how prac-ticing scientists and social scientists make the distinction between mainstream andfringe, see Harry Collins, Rejecting Knowledge Claims Inside and Outside Science, 44SOC. STUDS. SCI. 722 (2014); Harry Collins, Andrew Bartlett & Luis Reyes-Galindo,Demarcating Fringe Science for Policy, 25 PERSPS. ON SCI. 411 (2017) (an earlier versionpromulgated as The Ecology of Fringe Science and its Bearing on Policy is available athttps://arxiv.org/abs/1606.05786 [https://perma.cc/HX8B-LP6A]).10. Some contemporary consequences are well-described by James Heathers,The Real Scandal About Ivermectin, THE ATLANTIC (Oct. 23, 2021), https://www.theatlantic.com/science/archive/2021/10/ivermectin-research-problems/620473/ [https://perma.cc/CF8Y-D7MU].  It has long been known that scienceworks by ignoring most of the literature, including peer-reviewed publications, butHeathers points out the quarantine with respect to what insiders know they shouldignore is violated at the time of a pandemic (quarantine is an apt metaphor), be-cause of the explosion of social media discussion:Much research is simply ignored by other scientists because it either looks“off” or is published in the wrong place.  A huge gray literature exists inparallel to reliable clinical research, including work published in low-quality or outright predatory journals that will publish almost anythingfor money.  Likewise, the authors of fabricated or heavily distorted paperstend to have modest ambitions:  The point is to get their work in printand added to their CV, not to make waves.  We often say these studies aredesigned to be “written but not read.”  . . .  In a pandemic, when thestakes are highest, the somewhat porous boundary between these publica-tion worlds has all but disappeared.  There is no gray literature now:  Eve-rything is a magnet for immediate attention and misunderstanding.  Anunbelievable, inaccurate study no longer has to linger in obscurity; it maybubble over into the public consciousness as soon as it appears online,and get passed around the internet like a lost kitten in a preschool.  Aninstantly forgettable preprint, which would once have been read by only afew pedantic experts, can now be widely shared among hundreds ofthousands on social media.Id. (citing Jean Adams, Frances C. Hillier-Brown, Helen J. Moore, Amelia A. Lake,Vera Araujo-Soares, Martin White & Carolyn Summerbell, Searching and Synthesising‘Grey Literature’ and ‘Grey Information’ in Public Health: Critical Reflections on Three CaseStudies, SYSTEMIC REVS. (Sept. 29, 2016), https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-016-0337-y [https://perma.cc/7LEE-DDPZ]).5Collins: The Owls: Some Difficulties in Judging Scientific ConsensusPublished by Villanova University Charles Widger School of Law Digital Repository, 2023882 VILLANOVA LAW REVIEW [Vol. 67: p. 877(3) There are a small number of corrupt scientists who publishcontrarian papers so as to sow doubt, seemingly for financialreward, so as to create fake scientific controversies.  Thereare other fake scientific controversies promulgated by scien-tists who are “past their sell-by” date but who can easily fooloutsiders.11  Scientific and social scientific understandingneeds to contribute to picking these things apart.(4) There are entire “fake” sciences, or sciences that seem not toaim for goals that sciences would normally try to attain.  Forinstance, in the legal setting it seems that fingerprint expertsare reluctant to expose their skills to blind testing, thoughthis would not be difficult—instead fingerprint identificationcontinues to be used in the courtroom as more of a “tradi-tion.”  Under modern understandings of science, it could beargued that mainstream economics is not really a science be-cause of the relative value it puts on maintaining what is evermore obviously a false model of human interaction, so it isno longer a search for truth, but it needs a secure under-standing of science and social science to say so.What all this means is that it is not going to be possible simply to switchthe criterion to consensus within the current timetable and ethos of thecourtroom.  But there has to be a better solution than the current gear-grinding of science and the law.  Cheng’s analysis must form the basis ofthe solution—one that moves the debate in the right direction.  It is now amatter of working our way toward institutions that will enable this to workin the context of the courtroom.11. See NAOMI ORESKES & ERIK M. CONWAY, MERCHANTS OF DOUBT: HOW AHANDFUL OF SCIENTISTS OBSCURED THE TRUTH ON ISSUES FROM TOBACCO SMOKE TOGLOBAL WARMING (2010) (discussing corrupt scientists); see also Collins & Evans,supra note 8 (discussing sell-by date and fake scientific controversies, initially basedon Martin Weinel’s investigation of South African President Thabo Mbeki’s refusalto distribute anti-retroviral drugs to pregnant women).6Villanova Law Review, Vol. 67, Iss. 5 [2023], Art. 3https://digitalcommons.law.villanova.edu/vlr/vol67/iss5/3

The Owls: Some Difficulties in Judging Scientific Consensus

bepress Legal Repository

https://digitalcommons.law.villanova.edu/cgi/viewcontent.cgi?article=3541&context=vlr

The Owls: Some Difficulties in Judging Scientific Consensus

Abstract

Similar works

Full text

Available Versions

Villanova University School of Law: Digital Repository

bepress Legal Repository