Evaluating advanced search interfaces using established information-seeking model

When users have poorly defined or complex goals search interfaces offering only keyword searching facilities provide inadequate support to help them reach their information-seeking objectives. The emergence of interfaces with more advanced capabilities such as faceted browsing and result clustering can go some way to some way toward addressing such problems. The evaluation of these interfaces, however, is challenging since they generally offer diverse and versatile search environments that introduce overwhelming amounts of independent variables to user studies; choosing the interface object as the only independent variable in a study would reveal very little about why one design out-performs another. Nonetheless if we could effectively compare these interfaces we would have a way to determine which was best for a given scenario and begin to learn why. In this article we present a formative framework for the evaluation of advanced search interfaces through the quantification of the strengths and weaknesses of the interfaces in supporting user tactics and varying user conditions. This framework combines established models of users, user needs, and user behaviours to achieve this. The framework is applied to evaluate three search interfaces and demonstrates the potential value of this approach to interactive IR evaluation

Discovery-led refinement in e-discovery investigations: sensemaking, cognitive ergonomics and system design.

Given the very large numbers of documents involved in e-discovery investigations, lawyers face a considerable challenge of collaborative sensemaking. We report findings from three workplace studies which looked at different aspects of how this challenge was met. From a sociotechnical perspective, the studies aimed to understand how investigators collectively and individually worked with information to support sensemaking and decision making. Here, we focus on discovery-led refinement; specifically, how engaging with the materials of the investigations led to discoveries that supported refinement of the problems and new strategies for addressing them. These refinements were essential for tractability. We begin with observations which show how new lines of enquiry were recursively embedded. We then analyse the conceptual structure of a line of enquiry and consider how reflecting this in e-discovery support systems might support scalability and group collaboration. We then focus on the individual activity of manual document review where refinement corresponded with the inductive identification of classes of irrelevant and relevant documents within a collection. Our observations point to the effects of priming on dealing with these efficiently and to issues of cognitive ergonomics at the human–computer interface. We use these observations to introduce visualisations that might enable reviewers to deal with such refinements more efficiently

A Nine Month Report on Progress Towards a Framework for Evaluating Advanced Search Interfaces considering Information Retrieval and Human Computer Interaction

This is a nine month progress report detailing my research into supporting users in their search for information, where the questions, results or even thei

Searching Data: A Review of Observational Data Retrieval Practices in Selected Disciplines

A cross-disciplinary examination of the user behaviours involved in seeking and evaluating data is surprisingly absent from the research data discussion. This review explores the data retrieval literature to identify commonalities in how users search for and evaluate observational research data. Two analytical frameworks rooted in information retrieval and science technology studies are used to identify key similarities in practices as a first step toward developing a model describing data retrieval

A framework for investigating the interaction in information retrieval

To increase retrieval effectiveness, information retrieval systems must offer better supports to users in their information seeking activities. To achieve this, one major concern is to obtain a better understanding of the nature of the interaction between a user and an information retrieval system. For this, we need a means to analyse the interaction in information retrieval, so as to compare the interaction processes within and across information retrieval systems. We present a framework for investigating the interaction between users and information retrieval systems. The framework is based on channel theory, a theory of information and its flow, which provides an explicit ontology that can be used to represent any aspect of the interaction process. The developed framework allows for the investigation of the interaction in information retrieval at the desired level of abstraction. We use the framework to investigate the interaction in relevance feedback and standard web search

Review of employment and skills: April 2011

"This Review has its foundation in the Leitch Report published in 2006, which recommended the development of an “integrated employment and skills service to help people meet the challenges of the modern labour market” and for the UK Commission for Employment and Skills to report on the changes required to deliver integrated services. The UK Commission’s 2010-11 Grant in Aid Letter required: “The continuation of a Review that has as its focus progress on integrating employment and skills systems”. This report covers England only. There will be separate reporting for Wales and Scotland after the elections in May 2011." - Page 5