Human institutions and activities cannot avoid failures. Anxiety about them often provokes governments to try to prevent those failures. When that anxiety is vivid and urgent, government may do so without carefully asking whether regulation’s costs justify their benefits. Privacy and Accountability in Black Box Medicine admirably labors to bring discipline and rationality to thinking about an important development — the rise of “black-box medicine” — before it causes injuries regulation should have prevented and before it is impaired by improvident regulation. That is, Privacy and Accountability weighs the costs against the benefits of various forms of regulation across the many kinds of black-box medicine

Schneider, Carl E.

English

bepress Legal Repository

A Comment on \u3cem\u3ePrivacy and Accountability in Black-Box Medicine\u3c/em\u3e

University of Michigan School of Law

Michigan Telecommunications and Technology Law ReviewVolume 23 | Issue 22017A Comment on Privacy and Accountability in Black-Box MedicineCarl E. SchneiderUniversity of Michigan Law School, carlschn@umich.eduFollow this and additional works at: http://repository.law.umich.edu/mttlrPart of the Health Law and Policy Commons, Privacy Law Commons, and the Science andTechnology Law CommonsThis Comment is brought to you for free and open access by the Journals at University of Michigan Law School Scholarship Repository. It has beenaccepted for inclusion in Michigan Telecommunications and Technology Law Review by an authorized editor of University of Michigan Law SchoolScholarship Repository. For more information, please contact mlaw.repository@umich.edu.Recommended CitationCarl E. Schneider, A Comment on Privacy and Accountability in Black-Box Medicine, 23 Mich. Telecomm. & Tech. L. Rev. 321(2017).Available at: http://repository.law.umich.edu/mttlr/vol23/iss2/2A COMMENT ON PRIVACY ANDACCOUNTABILITY IN BLACK-BOX MEDICINECarl E. SchneiderCite as: Carl E. Schneider, Comment,A Comment on Privacy and Accountability in Black-Box Medicine,23 MICH. TELECOM. & TECH. L. REV. 321 (2017).This manuscript may be accessed online at repository.law.umich.edu.Human institutions and activities cannot avoid failures.  Anxiety aboutthem often provokes governments to try to prevent those failures.  When thatanxiety is vivid and urgent, government may do so without carefully askingwhether regulation’s costs justify their benefits. Privacy and Accountabilityin Black Box Medicine admirably labors to bring discipline and rationality tothinking about an important development — the rise of “black-boxmedicine” — before it causes injuries regulation should have prevented andbefore it is impaired by improvident regulation.  That is, Privacy and Ac-countability weighs the costs against the benefits of various forms of regula-tion across the many kinds of black-box medicine.Privacy and Accountability’s enterprise is rewarding because (as the ar-ticle says) black-box medicine offers major hopes for much progress inmany fields. Privacy and Accountability is exemplary in identifying earlythe range of reasons for and against regulating black-box medicine and therange of ways it might be done adroitly.  This task, of course, is as dauntingas it is desirable:  Identifying the costs and benefits of black-box medicineand of its regulatory alternatives is hardly possible, if only because so manysorts of elusive data are needed.  However, a more searching and skepticalinquiry might advance the article’s search for systematic, disciplined, realis-tic, and thus (I believe) cautious ways to regulate black-box medicine.First, the ardor with which we should regulate black-box medicine de-pends, as the article says, on how much good it can do.  The authors ac-knowledge that the quantum of good is some form of “a lot,” but theydescribe the good blandly — as “innovation.”  That anodyne term hardlybrings home what is at stake.  If black-box medicine attains anything like thearticle’s expectations, many people will live who would otherwise die, manypeople will enjoy health who would otherwise endure disease, and manypeople will escape suffering who would otherwise bear it.  A clearer, fullerstatement of the promise of black-box medicine might better reveal howmuch death, disease, and suffering it might diminish and thus what over-regulation might cost.Second, regulation gains allure if the regulatory choice is between a safestatus quo (current treatments that work) and risky future (innovations that321322 Michigan Telecommunications and Technology Law Review [Vol. 23:321might fail).  The authors are too sophisticated to think this is actually ourchoice, but it’s how they often appear to talk and what many people clearlythink.  Yet if black box algorithms are blocked until regulators are satisfied,algorithms will ordinarily not go away, for health-care decisions generallyrest on algorithms of some sort.  Many algorithms go unexamined and slipinsidiously into medicine under, for example, rubrics like “my clinical expe-rience.”  Even examined algorithms are often unsupported by satisfactoryevidence (if only because they are the best our present ignorance permits).Third, arguments for regulation too often seem to assume that regulatorswill work acutely and wisely. Privacy and Accountability could usefully askmore rigorously whether regulation will correctly identify good and bad al-gorithms and products and then properly sort them.  If, as the article vigor-ously argues, evaluating these algorithms and products is nastily hard, howaccurate can regulators be?  How well have similar regulators — like theFDA and the system of Institutional Review Boards that I will shortly dis-cuss — done separating wheat from chaff?Fourth, and critically, the article does not really undertake the (admit-tedly arduous) work of assessing the actual risks of black-box medicine andthus may make too much of its special dangers.  Consider the article’s omi-nous account of privacy risks.  Although the privacy literature is massive,we are not proffered evidence about the actual privacy consequences ofblack-box medicine.  Rather, we are vaguely warned that “losses of patientprivacy . . . can cause significant harms” and that “people consider healthinformation to be especially sensitive.”  The article then lists four harmsbreaches of medical privacy can inflict.  The article does not, however, esti-mate how often medical privacy is breached or show that “breaches” harmpatients.Despite the lack of evidence (or even speculation) about such funda-mental questions, in its crucial Part IV (“Reconciling Privacy and Accounta-bility”) the article repeatedly advocates often-restrictive regulation becauseprivacy concerns are weighty. The article advocates “a system of specificrestrictions on the collection, use, or disclosure of patient health informa-tion” because “the privacy interests . . . are so great.”  [39]  The article advo-cates “independent gatekeepers” because “the privacy interests . . . are sogreat.”  [43].  Among the reasons for updating regulations “continuously” isthat the cost of privacy breaches is “so high.”  And the article wants spe-cially stringent “safeguards” because “the potential privacy harms . . . are sogreat.”  [48]The absence of evidence is particularly bothersome because medical re-search evaluating evaluate black-box medicine — or most other kinds ofmedicine — cannot easily cause damaging breaches of privacy.  In mostresearch “the risk of inadvertent or damaging disclosure of sensitive infor-Spring 2017] A Comment on Privacy and Accountability in Black-Box Medicine 323mation is extremely low.”1  Long before HHS blessed us with HIPAA, asurvey of over two thousand research projects found only three confidential-ity breaches “that had harmed or embarrassed a subject.  And of 729 behav-ioral projects, only four reported ‘harmful effects.’ ”2  Richard Campbellcould find “very few” confidentiality breaches.3  Green knew of “no in-stances of harm from unauthorized disclosures” in observational health-ser-vices research.  Georgetown’s Health Privacy Project did not document any.4An HHS advisory committee found no confidentiality breaches in research-ers’ use of records.5Harm, after all, hangs on an attenuated chain of events:  Informationpeople want secret must be collected, but most research does not do so.  Theinformation must be revealed, but researchers have few incentives to releaseit and many reasons not to.  The information must reach someone able andwilling to misuse it.  Who is that?  The modal fear is insurance companiesusing genetic information to deny people coverage. But doing so is illegal,and Hall and Rich report that there has not been and is unlikely to be “seri-ous adverse selection” and that “[m]ultiple, independent sources refuted” orcould not “document, any substantial level of genetic discrimination.”6  An-other study found no evidence of insurers cancelling policies in this way.7This actually makes sense.  A researcher would need a lot of informa-tion about the insureds of a particular company.  It would have to be reliableinformation about factors significantly affecting insurability (and most ge-netic information is low on the list).  Furthermore, confidentiality is an oldprinciple of medicine and research.  An AAUP subcommittee found privacyconcerns” entirely met by long-standing departmental and disciplinary prac-tice.”8  Furthermore, confidentiality is an old legal duty.  The Association ofAcademic Health Centers argues that HIPAA may offer no “greater protec-1. Academy of Medical Sciences, Personal Data for Public Good: Using Health Infor-mation in Medical Research 41 (Jan 2006), http://www.acmedsci.ac.uk/viewFile/publicationDownloads/Personal.pdf.2. Zachary M. Schrag, How Talking Became Human Subjects Research: The FederalRegulation of the Social Sciences, 1965-1991, 21 J Policy History 3, 29 (2009).3. Richard T. Campbell, Risk and Harm Issues In Social Science Research, HumanSubjects Policy Conference at 7 - 8 (April 2003).4. Lee A. Green et al, IRB and Methodological Issues: Impact of Institutional ReviewBoard Practice Variation on Observational Health Services Research, 41 HSR: Health Ser-vices Research 214, 225 (2006).5. William Burman & Robert Daum, Grinding to a Halt: The Effects of the IncreasingRegulatory Burden on Research and Quality Improvement Efforts, 49 Clinical Infectious Dis-eases 328, 329 (2009).6. Mark A. Hall & Stephen S. Rich, Laws Restricting Health Insurers’ Use of GeneticInformation: Impact on Genetic Discrimination, 66 American J Human Genetics 293, 305(2000).7. Karen J. Wingrove et al, Experiences and Attitudes Concerning Genetic Testing andInsurance in a Colorado Population: A Survey of Families Diagnosed With Fragile X Syn-drome, 64 American J Medical Genetics 378, 380 (1996).8. AAUP, Research on Human Subjects at 4.324 Michigan Telecommunications and Technology Law Review [Vol. 23:321tion than current longstanding effective regulation,”9 like FDA and NIHrules and state privacy law.  Before HIPAA there was no “explosion of im-proper disclosures” or any “systematic unwillingness to deal with theproblems that do arise.”10 HHS’s rationale for HIPAA centrally relied onnine anecdotes, like health-insurance claims forms blowing out of a truck oran employee stealing a health-department disk.  Unfortunate events, butwhile laws already penalize negligence and theft, winds still blow and em-ployees still steal.Nor is it clear that black-box medicine touches unusually sensitive pri-vacy questions.  Some people, for example, are voluble about their abortionsbut taciturn about their money.  Releasing the information used in identitythefts harms many people much more than disclosing medical information.And while if asked if they value privacy people say “yes,” they are worthilywilling to risk privacy to promote research.  For example, in one “commu-nity-based primary care practice, only 3.6% of the patients refused to allowtheir medical records to be used in research.”11  Even groups widely thoughtto distrust medical researchers — like African Americans and rural Africans— generously contribute information to research.12Privacy and Accountability argues that black-box medicine specially en-dangers privacy because it is so greedy for data:  “The more informationthat’s collected, disclosed, and used, the greater the raw material for futureprivacy problems and the more serious those problems are likely to be whenthey do occur.”  [38.]  This is logically plausible but experientially implausi-ble.  Indeed, as the article acknowledges, the reverse may be true.  The floodof information needed may make damaging use of private information lesslikely, since fishing a datum out of flood and then finding some way toabuse it are so hard.  Researchers working with such floods rarely care aboutany individual’s droplet.  What matters is the large numbers, not single datapoints.Privacy and Accountability buttresses its argument for “accountability”for black-box medicine because its products are “classic credence goods.”Perhaps its consumer products are, but the rest are ordinarily used or dis-pensed by doctors, who should not take such things on faith. Privacy andAccountability also wants accountability because black-box medicine is spe-cially hard to evaluate.  Is it really?  Much medical thinking has been dis-9. Association of Academic Health Centers Press Release, HIPAA Creating Barriersto Research and Discovery: HIPAA Problems Widespread and Unresolved Since 2003, Asso-ciation of Academic Health Centers 1, 6 (June 16, 2008).10. Richard A. Epstein, HIPAA on Privacy:  Its Unintended and Intended Conse-quences, 22 Cato J 13, 23 (2002).11. Barbara P. Yawn et al, The Impact of Requiring Patient Authorization for Use ofData in Medical Records Research, 47 J Family Practice 361, 363 (1998).12. E.g., David Wendler et al, Research on Stored Biological Samples: The Views ofUgandans, 27 BMJ 1, 1 (2005).Spring 2017] A Comment on Privacy and Accountability in Black-Box Medicine 325credited in recent decades, and much treatment is unsupported by decentevidence.  Is black-box medicine so different that it needs special scrutiny?True, algorithms can be “so complex as to defy understanding.”  But howmuch understanding is needed?  Theories about how the body and medicinework are notoriously contestable, and the real test of medicine is its effect(whatever its theory).Finally, Privacy and Accountability does not explain why its elaboratearray of regulatory devices is necessary when so many of them are alreadylaw and when there are regulators the article generally excludes from itsanalysis.  The article focuses on only two basic regulators — the FDA andpatients, providers, and insurers.  But the enterprises that produce algorithmsare also regulated by, for instance, tort law and the market (which the articlediscusses obliquely in examining providers and insurers).  Given the calami-tous consequences of lost tort suits and of bad publicity about products, it’shard to regulate well without accounting for tort law and market forces.Nor is it clear why researchers are not included in the list of evaluatorsof black-box medicine.  They actually do the evaluative work, and their ca-reers depend on doing it competently.  There are certainly many problemswith researchers’ work, but those are problems that presumably apply toresearch about all kinds of medicine, not just black-box medicine.Finally, there is already extensive government regulation in many of theareas the article addresses.  As I have intimated, medical privacy is protectedby so many kinds of government agencies and rules that some of them —HIPAA, for instance — provoke bitter humor from physicians, patients, andresearchers.  Not least, most of the regulatory methods Privacy and Account-ability canvasses are already used by the Institutional Review Boards thatapprove vast stretches of human-subject research.  IRBs have regulated thatresearch — and especially the privacy concerns it raises — so fiercely and,it must be said, foolishly13 that it is hard to see why black-box medicineshould be entangled in further restrictions and why we should believe thatthe new regulators Privacy and Accountability anticipates will do better.Privacy and Accountability stands out in the literature on regulatingmedical research because it is conspicuously more willing to weigh the ben-efits of research against its costs, because it is conspicuously more sensitiveto the breadth of both medical research and regulatory methods, and becauseit is conspicuously less bound by the dogmas of the literature.  Because itsucceeds so estimably in these essential ways, I hope that the authors con-tinue working yet more intensively and freshly along the lines that they haverichly opened.13. For a thorough analysis of this conclusion, see Carl E. Schneider, The Censor’sHand:  The Misregulation of Human-Subject Research (MIT Press, 2015).

https://repository.law.umich.edu/cgi/viewcontent.cgi?article=1227&amp;context=mttlr

A Comment on \u3cem\u3ePrivacy and Accountability in Black-Box Medicine\u3c/em\u3e

Abstract

Similar works

Full text

Available Versions

bepress Legal Repository

University of Michigan School of Law