Evaluating Advanced Search Interfaces using Established Information-Seeking Models

When users have poorly defined or complex goals, keyword searching may not provide sufficient support. Subsequently, more advanced systems are being developed to support richer modes of search. This paper presents a formative framework for evaluating advancing search systems, which are providing more versatile environments that become increasingly hard to compare. This is done by quantifying their strengths and weaknesses in supporting user tactics and varying user conditions. This framework combines established models of users, user needs, and user behaviours to achieve this. The framework is applied to evaluate three advanced search interfaces and shows promising results

Evaluating advanced search interfaces using established information-seeking model

When users have poorly defined or complex goals search interfaces offering only keyword searching facilities provide inadequate support to help them reach their information-seeking objectives. The emergence of interfaces with more advanced capabilities such as faceted browsing and result clustering can go some way to some way toward addressing such problems. The evaluation of these interfaces, however, is challenging since they generally offer diverse and versatile search environments that introduce overwhelming amounts of independent variables to user studies; choosing the interface object as the only independent variable in a study would reveal very little about why one design out-performs another. Nonetheless if we could effectively compare these interfaces we would have a way to determine which was best for a given scenario and begin to learn why. In this article we present a formative framework for the evaluation of advanced search interfaces through the quantification of the strengths and weaknesses of the interfaces in supporting user tactics and varying user conditions. This framework combines established models of users, user needs, and user behaviours to achieve this. The framework is applied to evaluate three search interfaces and demonstrates the potential value of this approach to interactive IR evaluation

A Nine Month Report on Progress Towards a Framework for Evaluating Advanced Search Interfaces considering Information Retrieval and Human Computer Interaction

This is a nine month progress report detailing my research into supporting users in their search for information, where the questions, results or even thei

Evaluating Collaborative Information Seeking Interfaces with a Search-Oriented Inspection Method and Re-framed Information Seeking Theory

Despite the many implicit references to the social contexts of search within Information Seeking and Retrieval research, there has been relatively little work that has specifically investigated the additional requirements for collaborative information seeking interfaces. Here, we re-assess a recent analytical inspection framework, designed for individual information seeking, and then apply it to evaluate a recent collaborative information seeking interface: SearchTogether. The framework was built upon two models of solitary information seeking, and so as part of the re-assessment we first re-frame the models for collaborative contexts. We re-frame a model of search tactics, providing revised definitions that consider known collaborators. We then re-frame a model of user profiles to analyse support for different group dynamics. After presenting an analysis of SearchTogether, we reflect on its accuracy, showing that the framework identified 8 known truths, 8 new insights, and no known-to-be-untrue insights into the design. We conclude that the framework a) can still be applied to collaborative information seeking interfaces; b) can successfully produce additional requirements for collaborative information seeking interfaces; and c) can successfully model different dynamics of collaborating searchers

Structured Inspections of Search Interfaces: A Practitioners Guide

In this paper we present a practitioners guide on how to apply a new inspection framework that evaluates search interfaces for their support of different searcher types. Vast amounts of money are being invested into search, and so it is becoming increasingly important to identify problems in design early, while it is relatively cheap to rectify them. The inspection method presented here can be applied quickly to early prototypes, as well as existing systems, and goes beyond other inspection methods, like Cognitive Walkthroughs, to produces rich analyses, including the support provided for different search tactics and user types. The guide is presented as a detailed example, assessing a previously unevaluated search interface: the Tabulator, and so also provides design recommendations for improving it. We conclude with a summary of the benefits of the evaluation framework, and discuss our plans for future enhancements

Users' trust in information resources in the Web environment: a status report

This study has three aims; to provide an overview of the ways in which trust is either assessed or asserted in relation to the use and provision of resources in the Web environment for research and learning; to assess what solutions might be worth further investigation and whether establishing ways to assert trust in academic information resources could assist the development of information literacy; to help increase understanding of how perceptions of trust influence the behaviour of information users

Web Portal Design Guidelines as Identified by Children through the Processes of Design and Evaluation

The Web is an important source of information for school projects, but young users do not always find it easy to locate relevant material. A critical factor in success is the portal through which they search or browse web content. Traditionally web portals have been designed by adults with young users in mind, but there is very little evidence that the latter make use of them. In this paper design guidelines are elaborated for such portals that are based upon focus group and operational evaluations by elementary school students of two prototype web portals designed by two intergenerational teams, each comprising elementary school students and adult designers. The evaluations offer strong support for involving children throughout the design process for portals that both in presentation and functionality reflect the cognitive and affective needs of young users rather than adults

From Keyword Search to Exploration: How Result Visualization Aids Discovery on the Web

A key to the Web's success is the power of search. The elegant way in which search results are returned is usually remarkably effective. However, for exploratory search in which users need to learn, discover, and understand novel or complex topics, there is substantial room for improvement. Human computer interaction researchers and web browser designers have developed novel strategies to improve Web search by enabling users to conveniently visualize, manipulate, and organize their Web search results. This monograph offers fresh ways to think about search-related cognitive processes and describes innovative design approaches to browsers and related tools. For instance, while key word search presents users with results for specific information (e.g., what is the capitol of Peru), other methods may let users see and explore the contexts of their requests for information (related or previous work, conflicting information), or the properties that associate groups of information assets (group legal decisions by lead attorney). We also consider the both traditional and novel ways in which these strategies have been evaluated. From our review of cognitive processes, browser design, and evaluations, we reflect on the future opportunities and new paradigms for exploring and interacting with Web search results