In an article intitled, “Epistemological Dead End and Ergonomic Disaster? The North Amercian Collections Inventory Project,” Journal of Academic Librarianship 13 (September 1987): 209-213, David Henige charges that academic libraries are deceiving themselves when they assign a number to a subject collection and claim that this number represents the strength of the collection.  Assigning numbers or levels to subject collections is the methodology used by the RLG Conspectus to rate academic collections.  This article by Davis and Saunders offers a counter argument, based on subjective probabilities, to justify the RLG procedure

Davis, Donald G., Jr.

Saunders, E. Stewart

English

Purdue E-Pubs

Purdue UniversityPurdue e-PubsLibraries Research Publications1-1-1992Decision Making and UncertaintyDonald G. Davis Jr.University of Texas at Austin, dgdavis@ischool.utexas.eduE. Stewart SaundersPurdue University, ssaunder@purdue.eduFollow this and additional works at: http://docs.lib.purdue.edu/lib_researchThis document has been made available through Purdue e-Pubs, a service of the Purdue University Libraries. Please contact epubs@purdue.edu foradditional information.Davis, Donald G. Jr. and Saunders, E. Stewart, "Decision Making and Uncertainty" (1992). Libraries Research Publications. Paper 19.http://docs.lib.purdue.edu/lib_research/19Copyrighted 1992 by Mountainside Publishing. Current Publisher is Elsevier Science.  David Henige has accused the library research community of claiming validity for research that, in fact, is no better than unjustified belief. Not only is most library research of this genre, claims Henige, but this fact may be clouded by tables of statistics that leave the reader comfortable in the notion that the conclusions are valid. He further claims that the research community has not been called upon to account for such shoddy research.  Qualitative Judgments and Self-Validation   Henige's case in point is the RLG Conspectus. According to him it is a perfect example of unjustified belief and the fallacious use of numbers and statistics. At the risk of putting words in Henige's mouth, we would like to restate the core of his argument. His first point is that numerical rankings of collections don't measure anything. What the Conspectus calls a "research level" is oranges, while what it calls an "instruction level" is apples. This being the case, any attempt to rank collections is subjective in the extreme. Any attempt to verify those rankings by counting the number of volumes in the collection is only begging the question. His second complaint is that verification attempts have been "selfvalidating" in that they have attempted to define the five collection levels by averaging the intra-level self-reported rankings, a process Henige terms "grading on the curve."1Having proposed this several years ago, Henige now finds it not just disappointing but disconcerting that no one has either supported or opposed him in print. This is just further evidence, he suggests, that the library research community is unconcerned with directly falsifying or substantiating claims about the validity of the Conspectus.  Library research does not go unchallenged, however, when it treats important topics. The Pittsburgh study by Allen Kent et al. and Michael Harris's  thesis on American public libraries drew a host of critics. True, the Conspectus has not elicited a flood of articles; but claims for and against its validity have been made. For instance, Paul Mosher and Nancy Gwinn have used a historical justification for the qualitative nature of the instrument.2 They argue that the quantitative collection evaluations of the 1950s and 1960s failed to produce usable results. Mosher also disputes Robert Downs' rationale for quantitative assessments. Two published articles have raised questions about the measurability of collection strengths. Stewart Saunders et al. published a study demonstrating the difficulty of identifying subject collections,3 and Ross Atkinson showed that "instruction level" and "research level" collections used two different measures.4  Henige does not agree that the RLG Collection Management and Development Committee had some legitimate reasons for including qualitative factors in the evaluation. From the Committee's point of view, a shelflist count contains many discrepancies due to varying practices among libraries. In addition, in most cases it does not account for newspaper, journal, microform, and manuscript collections. The alternative to a quantitative measure is by definition qualitative-in this case the judgment of an expert. Henige might allow that an assessment of a collection's quality is a desirable goal, but he would also argue that existing quality assessments are too subjective to have any meaning. He points out that libraries with similar collections sometimes arrive at widely divergent ratings of their strength levels. Henige is correct in noting that having rejected an absolute quantitative evaluation, it makes no sense to adopt selected quantitative measures to verify the judgment in any absolute sense.  Subjectivity and Probability    Henige has found the whole effort of subjectively evaluating the strengths of a  collection to be messy and full of imponderables, and therefore of highly dubious value. But is this the same thing as saying that the method lacks epistemological validity?  In the everyday world we make decisions in situations of uncertainty. We make those decisions by assigning a subjective probability to the likelihood that a given proposition is true or false. In the field of statistics, objective probabilities are determined through repeated trials, but most of life's situations do not allow for repeated trials. In most situations people assign subjective probabilities simply in order to act. The United States is out of the economic recession of 1990/91, true or false? There are multiple measures of economic recessions, but economists set discount rates based on their subjective assessment of the probability that this statement is true. Leonard Savage has shown that under appropriate conditions these subjective probabilities have many of the features of objective probabilities.5 For instance, he has demonstrated that it is possible to make a subjective but valid assertion that P(A)>P(B)-that is, the probability of A is greater than the probability of B-just from the conditions of the situation. In the situation under discussion, it is possible to state (without knowing either the subject of a collection or the library at which it is located) that P(level=2 or level=3)>P(level=1 or level=4)>P(level=0 or level=5) just from the internal logic of the Conspectus. Once there is some knowledge about the collection, these probabilities can be re-vised. Verification studies do not really verify anything; rather they help us to revise the probability of a prior assessment. This is the standard procedure used in statistical decision theory.  Probabilistic belief as a foundation for knowledge is not too different from the “justified true belief” theory cited by Henige. It requires an honest effort on the part of the believer and a rational weighing of the evidence. But let's look at the belief on which the believer is weighing odds; in what sense can it be true or false if it cannot be defined? No doubt Henige would argue that since the essence of what it means "to be a level 4" cannot be defined, then to state that "such and such a collection is a level 4" has no truth value. A proposition must have a truth value in order to use subjective probabilities.   Meeting Needs in the Real World   Collection strengths, like many other entities in society, are culturally defined. They have a certain definition by the consent of the community. Ross Atkinson states that the numerical rankings in the Conspectus statements are initially meaningless symbols. He uses the term autoclarification to describe how over time they gradually acquire meaning through use.6 When a bibliographer asserts, "the collection on medieval architecture is a level 4," he or she asserts "I have a high degree of confidence (probability) that were my colleagues at other institutions to evaluate this collection, they would be in general agreement that it merits a level 4." That is all the Conspectus claims to say or can say.  Readers may remember the collapse of the farm credit system a number of years ago. Few, if any, agricultural economists predicted it. Some observers commented that the economists were all too busy running their closed logic models on computers to be aware of events in the real world. The Conspectus is an attempt to meet the needs of cooperative collection building in the real world. It is legitimate for a kibitzer to critique the Conspectus from the vantage of a closed model, but those who are responsible for meeting the needs of collection development must make decisions in situations of uncertainty.  References  1.  David Henige, "Epistemological Dead End and Ergonomic Disaster? The North American Collections Inventory Project," Journal of Academic Librarianship 13 (September 1987): 209-213.  2.  Paul H. Mosher, "Collection Evaluation in Research Libraries: The Search for Quality, Consistency, and System in Collection Development," Library Resources and Technical Services 23 (Winter 1979): 16-32; Nancy E. Gwinn and Paul H. Mosher, "Coordinating Collection Development: The RLG Conspectus," College & Research Libraries 44 (March 1983): 128-140.  3.  Stewart Saunders, Harriet Nelson, and Priscilla Geahigan," Alternatives to the Shelflist Measure in Determining the Size of a Subject Collection," Library Research 3 (Winter 1981): 383-392.  4.  Ross Atkinson, "The Language of the Levels: Reflections on the Communication of Collection Development Policy," College & Research Libraries 47 (March 1986): 140-149.   5.  Leonard J. Savage, The Foundations of Statistics, N.Y.: Wiley, 1954. For a good nonmathematical treatment of subjective probabilities see the articles, "Coherence" and "Degrees of Belief' in the Encyclopedia of Statistical Sciences, N.Y.: Wiley, 1982.  6.  Atkinson, "The Language of the Levels," p. 141.   

Decision Making and Uncertainty

https://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=1022&amp;context=lib_research

Decision Making and Uncertainty

Abstract

Similar works

Full text

Available Versions

Purdue E-Pubs