MEASURING ONLINE SEARCH EXPERTISE

Search expertise has long been studied and used extensively in information seeking behavior research, both as a fundamental concept and as a method of comparing groups of users. Unfortunately, while search expertise has been studied for some time, the conceptualization of it has lagged behind its use in categorizing users. This has led to users who were defined as experts in one study who could be considered novices in another study. Not only does this make it difficult to know how search expertise impacts the issues being studied, it also makes it difficult to compare results between studies. It is clear that search expertise is more important now than ever as the information and misinformation available online grows exponentially. It must first be conceptually designed and modeled, and then it must be operationalized so that it can be reliably measured. This research first examined prior research related to online search expertise and created a working definition and model. One-on-one interviews were then conducted with nine known search experts, who were asked to describe online search expertise. These same experts were then gathered into three separate focus groups where they examined and grouped the items gathered from their individual interviews. The items and groupings from the focus groups were then used to update the model and also to create an initial instrument to measure online search expertise. That initial instrument was then given to 14 targeted participants in one-on-one cognitive interviews. The instrument was modified based upon the results of those interviews and then given to four targeted groups of participants and the results from 466 of those participants were examined using statistical methods. The results support the use of aggregate scores for past experience, self-rated search ability, and search skill ability as continuous measures of online search expertise. While the personality items used in the research suffered from the same inconclusive results as prior research, the inclusion of analytical abilities in future versions is indicated.Doctor of Philosoph

A multiple case study exploration of undergraduate subject searching

Subject searching—seeking information with a subject or topic in mind—is often involved in carrying out undergraduate assignments such as term papers and research reports. It is also an important component of information literacy—the abilities and experiences of effectively finding and evaluating, and appropriately using, needed information—which universities hope to cultivate in undergraduates by the time they complete their degree programs. By exploring the subject searching of a small group of upper-level, academically successful undergraduates over a school year I sought to acquire a deeper understanding of the contexts and characteristics of their subject searching, and of the extent to which it was similar in quality to that of search and domain experts. Primary data sources for this study comprised subject searching diaries maintained by participants, and three online subject searches they demonstrated at the beginning, middle, and end of the study during which they talked aloud while I observed, followed by focused interviews. To explore the quality of study participants’ subject searching I looked for indications of advanced thinking in thoughts they spoke aloud during demonstration sessions relating to using strategy, evaluating, and creating personal understanding, which represent three of the most challenging and complex aspects of information literacy. Applying a layered interpretive process, I identified themes within several hundred instances of participants’ advanced thinking relating to these three information literacy elements, with evaluative themes occurring most often. I also noted three factors influencing the extent of similarity iii between the quality of participants’ advanced thinking and that of search and domain experts which reflected matters that tended to be i) pragmatic or principled, , ii) technical or conceptual, and iii) externally or internally focused. Filtered through these factors, participants’ instances of advanced thinking brought to mind three levels of subject searching abilities: the competent student, the search expert, and the domain expert. Although relatively few in number, I identified at least some advanced thinking evincing domain expert qualities in voiced thoughts of all but one participant, suggesting the gap between higher order thinking abilities of upper-level undergraduates and information literate individuals is not always dauntingly large.Ye

TREC interactive with Cheshire II

This paper describes the UC Berkeley's participation in the TREC-6, 7 and 8 interactive track experiments. In these three studies 24 searchers (four in TREC-6, eight in TREC-7, and 12 in TREC-8) conducted a total of 160 searches, half on the Cheshire II system and the other half on the ZPRISE system. In TREC-7 and TREC-8 questionnaires were administered to gather information about basic demographic and searching experience, about each search, about each of the systems, and ®nally, about the user's perceptions of the systems. In this paper I will brie¯y describe the systems used in the study and how they di€er in design goals and implementation. The results of the interactive track evaluations and the information derived from the questionnaires are then discussed and plans for further research are considered. Ó 200