The evaluation of ontologies: Editorial review vs. democratic ranking

Increasingly, the high throughput technologies used by biomedical researchers are bringing about a situation in which large bodies of data are being described using controlled structured vocabularies—also known as ontologies—in order to support the integration and analysis of this data. Annotation of data by means of ontologies is already contributing in significant ways to the cumulation of scientific knowledge and, prospectively, to the applicability of cross-domain algorithmic reasoning in support of scientific advance. This very success, however, has led to a proliferation of ontologies of varying scope and quality. We define one strategy for achieving quality assurance of ontologies—a plan of action already adopted by a large community of collaborating ontologists—which consists in subjecting ontologies to a process of peer review analogous to that which is applied to scientific journal articles

Overcoming Expert Disagreement In A Delphi Process. An Exercise In Reverse Epistemology

Disagreement among experts is a central topic in social epistemology. What should an expert do when confronted with the different opinion of an epistemic peer? Possible answers include the steadfast view (holding to one’s belief), the abstemious view (suspending one’s judgment), and moderate conciliatory views, which specify criteria for belief change when a peer’s different opinion is encountered. The practice of Delphi techniques in healthcare, medicine, and social sciences provides a real-life case study of expert disagreement, where disagreement is gradually transformed into consensus. An analysis of Delphi shows that moderate conciliatory views are descriptively more adequate than rival views. However, it also casts doubt on whether the debate in social epistemology is explanatory relevant vis-à-vis real life cases of expert disagreement, where consensus replaces truth, and acceptance is more explanatorily relevant than belief

QUESTIONING THE ADMISSIBILITY OF NONSCIENTIFIC TESTIMONY AFTER DAUBERT: THE NEED FOR INCREASED JUDICIAL GATEKEEPING TO ENSURE THE RELIABILITY OF ALL EXPERT TESTIMONY

This article examines the difficulty of finding a proper standard for evaluating non-scientific expert testimony. It analyzes the legal standard for the admission of expert testimony as set out in the Federal Rule of Evidence and the Daubert case. It reviews a split in courts as to how to apply these standards to non-scientific expert testimony. It ends with some proposals for the application of Daubert to non-scientific expert testimony and suggests an amendment to the Federal Rules of evidence

The Critical Role of Statistics in Demostrating the Reliability of Expert Evidence

Federal Rule of Evidence 702, which covers testimony by expert witnesses, allows a witness to testify “in the form of an opinion or otherwise” if “the testimony is based on sufficient facts or data” and “is the product of reliable principles and methods” that have been “reliably applied.” The determination of “sufficient” (facts or data) and whether the “reliable principles and methods” relate to the scientific question at hand involve more discrimination than the current Rule 702 may suggest. Using examples from latent fingerprint matching and trace evidence (bullet lead and glass), I offer some criteria that scientists often consider in assessing the “trustworthiness” of evidence to enable courts to better distinguish between “trustworthy” and “questionable” evidence. The codification of such criteria may ultimately strengthen the current Rule 702 so courts can better distinguish between demonstrably scientific sufficiency and “opinion” based on inadequate (or inappurtenant) methods