The Proficiency of Experts

Expert evidence plays a crucial role in civil and criminal litigation. Changes in the rules concerning expert admissibility, following the Supreme Court\u27s Daubert ruling, strengthened judicial review of the reliability and the validity of an expert\u27s methods. Judges and scholars, however, have neglected the threshold question for expert evidence: whether a person should be qualified as an expert in the first place. Judges traditionally focus on credentials or experience when qualifying experts without regard to whether those criteria are good proxies for true expertise. We argue that credentials and experience are often poor proxies for proficiency. Qualification of an expert presumes that the witness can perform in a particular domain with a proficiency that non-experts cannot achieve, yet many experts cannot provide empirical evidence that they do in fact perform at high levels of proficiency. To demonstrate the importance ofproficiency data, we collect and analyze two decades of proficiency testing of latent fingerprint examiners. In this important domain, we found surprisingly high rates of false positive identifications for the period 1995 to 2016. These data would qualify the claims of many fingerprint examiners regarding their near infallibility, but unfortunately, judges do not seek out such information. We survey the federal and state case law and show how judges typically accept expert credentials as a proxy for proficiency in lieu of direct proof of proficiency. Indeed, judges often reject parties\u27 attempts to obtain and introduce at trial empirical data on an expert\u27s actual proficiency. We argue that any expert who purports to give falsifiable opinions can be subjected to proficiency testing and that proficiency testing is the only objective means of assessing the accuracy and reliability ofexperts who rely on subjective judgments to formulate their opinions (so-called black-box experts ). Judges should use proficiency data to make expert qualification decisions when the data is available, should demand proof of proficiency before qualifying black-box experts, and should admit at trial proficiency data for any qualified expert. We seek to revitalize the standard for qualifying experts: expertise should equal proficiency

Cultural Bias in Judicial Decision Making

This Essay describes the phenomenon of cultural bias in judicial decision making, and examines the use of testimonies and opinions of cultural experts as a way to diminish this bias. The Essay compares the legal regimes of the United States and Israel. Whereas in the United States, the general practice of using cultural experts in courts is well developed and regulated, the Israeli legal procedure has no formal method for admitting cultural expert testimony, and examples of opinions or testimonies of cultural experts in the Israeli legal system are sporadic. The Essay further argues that social science evidence is an essential but insufficient means of reducing the cultural bias of judges. Judges’ reliance on cultural experts can also be fueled by a preexisting cultural agenda disguised as an informed judgment. The Essay concludes with a suggestion of measures that can be implemented alongside the use of cultural experts in order to increase judges’ awareness of the cultural bias and mitigate its consequences

Appropriateness of colorectal cancer screening: appraisal of evidence by experts

Objectives. To evaluate how the level of evidence perceived by an international panel of experts was concordant with the level of evidence found in the literature, to compare experts perceived level of evidence to their appropriateness scores, and to compare appropriateness criteria for colonoscopy between experts and an evidence-based approach. Design. Comparison of expert panel opinions and systematic literature review regarding the level of evidence and appropriateness of colonoscopy indications. Participants. European Panel on the Appropriateness of Gastrointestinal Endoscopy multidisciplinary experts from 14 European countries. Main outcome measures. Concordance and weighted kappa coefficient between level of evidence as perceived by the experts' and that found in the literature, and between panel- and literature-based appropriateness categories. Results. Experts overestimated the level of published evidence of 57 indications. Concordance between the level of evidence perceived by the experts and the actual level of evidence found in the literature was 36% (weighted kappa 0.18). Indications for colonoscopy were reported to be appropriate, uncertain, and inappropriate by the experts in 54, 19, and 27% of the cases, and by the literature in 37, 46, and 17% of the cases. A 46% agreement (weighted kappa 0.29) was found between literature-based and experts' appropriateness criteria. Conclusions. Experts often overestimated the level of evidence on which they based their decisions. However, rarely did the experts' judgement completely disagree with the literature, although concordance between panel- and literature-based appropriateness was only fair. A more explicit discussion of existing evidence should be undertaken with the experts before they evaluate appropriateness criteri

Beliefs, Credence Goods and Information Campaigns

We study the role of beliefs about experts' honesty in a market for credence goods with second opinions and overtreatment. Experts are honest or dishonest. The population shares a common belief about the share of honest experts, which may be incorrect. We characterize the belief that maximizes consumer's expected utility and show that it is generically different from the true share of honest experts and larger than the one that maximizes the equilibrium level of honesty. We then analyze the decision of an authority that has learned the actual share of honest experts whether to publicly reveal it through a costless information campaign, thus correcting people's beliefs, and show that it does not depend on how wrong beliefs are. We further show how increasing market transparency (making experts more aware of the number of opinions collected) affects the optimal belief and may have a positive as well as a negative effect on overtreament. Finally, we briefly see how a successful campaign run in Switzerland in the mid '80s to reduce excessive hysterectomy rates can be read through the lenses of our model and how accounting for beliefs about honesty might allow theoretical predictions to better fit experimental evidence

Importance of Different Types of Prior Knowledge in Selecting Genome‐Wide Findings for Follow‐Up

Biological plausibility and other prior information could help select genome‐wide association ( GWA ) findings for further follow‐up, but there is no consensus on which types of knowledge should be considered or how to weight them. We used experts’ opinions and empirical evidence to estimate the relative importance of 15 types of information at the single‐nucleotide polymorphism ( SNP ) and gene levels. Opinions were elicited from 10 experts using a two‐round Delphi survey. Empirical evidence was obtained by comparing the frequency of each type of characteristic in SNP s established as being associated with seven disease traits through GWA meta‐analysis and independent replication, with the corresponding frequency in a randomly selected set of SNP s. SNP and gene characteristics were retrieved using a specially developed bioinformatics tool. Both the expert and the empirical evidence rated previous association in a meta‐analysis or more than one study as conferring the highest relative probability of true association, whereas previous association in a single study ranked much lower. High relative probabilities were also observed for location in a functional protein domain, although location in a region evolutionarily conserved in vertebrates was ranked high by the data but not by the experts. Our empirical evidence did not support the importance attributed by the experts to whether the gene encodes a protein in a pathway or shows interactions relevant to the trait. Our findings provide insight into the selection and weighting of different types of knowledge in SNP or gene prioritization, and point to areas requiring further research.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/96262/1/gepi21705.pd

The Constitutional Regulation of Forensic Evidence

The Constitution increasingly regulates the use of forensic evidence in criminal cases. This is a remarkable shift. In decades past, the U.S. Supreme Court declined to provide strong due process protection against destruction of forensic evidence or to obtain defense access to experts. In contrast, in recent years, the Court’s series of Confrontation Clause rulings tightened requirements to present live testimony in the courtroom. Perhaps far more significant, I will argue, the Court has strengthened obligations of defense counsel to litigate forensics, twice underscoring in little noticed opinions: “Criminal cases will arise where the only reasonable and available defense strategy requires consultation with experts or introduction of expert evidence.” In this Essay, I describe how despite decades of missed opportunities to adequately regulate forensics, in recent rulings the Court and to a far greater degree, lower state and federal courts, increasingly focus on sound litigation of forensics. In an era of plea bargaining, the accuracy of forensic analysis depends far less on cross-examination at trial, and far more on sound lab techniques, full disclosure of strengths and limitations of forensic evidence to prosecutors and the defense, and careful litigation. The changing judicial understanding of the constitutional significance of forensic evidence in criminal cases may follow from a new appreciation that forensic evidence is not only increasingly important in criminal cases, but that many traditional techniques lack adequate reliability and validity. The Sixth Amendment and the Due Process Clauses are emerging as promising constitutional sources for improved regulation of forensics, including through ineffective assistance of counsel and Brady v. Maryland rulings focusing on investigations and plea bargains. How meaningful courts will make those dual constitutional protections in the years to come will be a crucial test of our commitment to accuracy in criminal justice

Issues concerning the Admissibility in Federal Courts of Business Records Containing Opinions or Diagnoses under Federal Rule of Evidence 803(6)

Federal Rule of Evidence 803(6) explicitly provides for the admissibility in the federal courts of business records containing opinions or diagnoses. Despite that, over the years, the courts have routinely excluded business records containing opinions and diagnoses for a variety of reasons. In 1979, for example, the Court of Claims fashioned a rule preventing the admission of expert appraisals unless the preparer of the appraisal were present to testify to his qualifications as an expert pursuant to Rules 702 and 705. Many courts continue to follow this rule. This note discusses and critiques that rationale for exclusion (and others), in light of more current jurisprudence concerning opinions, experts, and expert opinions

Expert Teaming – Bridging the Divide Between Party-Appointed and Tribunal Appointed Experts

It appears to be a given in international arbitration that expert evidence is provided by party-appointed experts. This is despite the fact that one increasingly hears complaints that such evidence is of little value because it advocates too much in the interests of the party presenting it and tribunals often have to decide between diametrically opposite opinions and irreconcilable conclusions. While techniques such as "pre-hearing meetings" and "witness conferencing" seek to address these concerns, international arbitral practice has shown that these techniques are not always sufficient to combat the disadvantages of party-appointed experts. Fewer efforts have been made to remove the concerns expressed with regard to tribunal-appointed experts. This article describes a technique which seeks to combine the advantages of party-appointed and tribunal-appointed experts. The technique has been successfully applied in several International Criminal Court arbitration proceedings and has become known as "expert teaming" as well as the "Sachs Protocol"

Lack of evidence does not justify neglect. how can we address unmet medical needs in calciphylaxis

Calcific uraemic arteriolopathy (CUA), or calciphylaxis, is a rare disease predominantly occurring in comorbidity with dialysis. Due to the very low frequency of CUA, prospective studies on its management are lacking and even anecdotal reports on treatment remain scarce. Therefore, calciphylaxis is still a challenging disease with dismal prognosis urgently requiring adequate strategies for diagnosis and treatment.In an attempt to fill some of the current gaps in evidence on various, highly debated and controversial aspects of dialysis-associated calciphylaxis, 13 international experts joined the 1st Consensus Conference on CUA, held in Leuven, Belgium on 21 September 2015. The conference was supported by the European Calciphylaxis Network (EuCalNet), which is a task force of the ERA-EDTA scientific working group on Chronic Kidney Disease-Mineral and Bone Disorders (CKD-MBD). After an intense discussion, a 9-point Likert scale questionnaire regarding 20 items on calciphylaxis was anonymously answered by each participant. These 20 items addressed unsolved issues in terms of diagnosis and management of calciphylaxis. On the one hand, the analysis of the expert opinions identified areas of general consensus, which might be a valuable aid for physicians treating such a disease with less experience in the field. On the other hand, some topics such as the pertinence of skin biopsy and administration of certain treatments revealed divergent opinions. The aim of the present summary report is to provide some guidance for clinicians who face patients with calciphylaxis in the current setting of absence of evidence-based medicin

“Can It Read My Mind?” – What Do the Public and Experts Think of the Current (Mis)Uses of Neuroimaging?

Emerging applications of neuroimaging outside medicine and science have received intense public exposure through the media. Media misrepresentations can create a gulf between public and scientific understanding of the capabilities of neuroimaging and raise false expectations. To determine the extent of this effect and determine public opinions on acceptable uses and the need for regulation, we designed an electronic survey to obtain anonymous opinions from as wide a range of members of the public and neuroimaging experts as possible. The surveys ran from 1st June to 30 September 2010, asked 10 and 21 questions, respectively, about uses of neuroimaging outside traditional medical diagnosis, data storage, science communication and potential methods of regulation. We analysed the responses using descriptive statistics; 660 individuals responded to the public and 303 individuals responded to the expert survey. We found evidence of public skepticism about the use of neuroimaging for applications such as lie detection or to determine consumer preferences and considerable disquiet about use by employers or government and about how their data would be stored and used. While also somewhat skeptical about new applications of neuroimaging, experts grossly underestimated how often neuroimaging had been used as evidence in court. Although both the public and the experts rated highly the importance of a better informed public in limiting the inappropriate uses to which neuroimaging might be put, opinions differed on the need for, and mechanism of, actual regulation. Neuroscientists recognized the risks of inaccurate reporting of neuroimaging capabilities in the media but showed little motivation to engage with the public. The present study also emphasizes the need for better frameworks for scientific engagement with media and public education