Evaluating Collaborative Information Seeking Interfaces with a Search-Oriented Inspection Method and Re-framed Information Seeking Theory

Despite the many implicit references to the social contexts of search within Information Seeking and Retrieval research, there has been relatively little work that has specifically investigated the additional requirements for collaborative information seeking interfaces. Here, we re-assess a recent analytical inspection framework, designed for individual information seeking, and then apply it to evaluate a recent collaborative information seeking interface: SearchTogether. The framework was built upon two models of solitary information seeking, and so as part of the re-assessment we first re-frame the models for collaborative contexts. We re-frame a model of search tactics, providing revised definitions that consider known collaborators. We then re-frame a model of user profiles to analyse support for different group dynamics. After presenting an analysis of SearchTogether, we reflect on its accuracy, showing that the framework identified 8 known truths, 8 new insights, and no known-to-be-untrue insights into the design. We conclude that the framework a) can still be applied to collaborative information seeking interfaces; b) can successfully produce additional requirements for collaborative information seeking interfaces; and c) can successfully model different dynamics of collaborating searchers

An analytical inspection framework for evaluating the search tactics and user profiles supported by information seeking interfaces

Searching is something we do everyday both in digital and physical environments. Whether we are searching for books in a library or information on the web, search is becoming increasingly important. For many years, however, the standard for search in software has been to provide a keyword search box that has, over time, been embellished with query suggestions, Boolean operators, and interactive feedback. More recent research has focused on designing search interfaces that better support exploration and learning. Consequently, the aim of this research has been to develop a framework that can reveal to designers how well their search interfaces support different styles of searching behaviour.The primary contribution of this research has been to develop a usability evaluation method, in the form of a lightweight analytical inspection framework, that can assess both search designs and fully implemented systems. The framework, called Sii, provides three types of analyses: 1) an analysis of the amount of support the different features of a design provide; 2) an analysis of the amount of support provided for 32 known search tactics; and 3) an analysis of the amount of support provided for 16 different searcher profiles, such as those who are finding, browsing, exploring, and learning. The design of the framework was validated by six independent judges, and the results were positively correlated against the results of empirical user studies. Further, early investigations showed that Sii has a learning curve that begins at around one and a half hours, and, when using identical analysis results, different evaluators produce similar design revisions.For Search experts, building interfaces for their systems, Sii provides a Human-Computer Interaction evaluation method that addresses searcher needs rather than system optimisation. For Human-Computer Interaction experts, designing novel interfaces that provide search functions, Sii provides the opportunity to assess designs using the knowledge and theories generated by the Information Seeking community. While the research reported here is under controlled environments, future work is planned that will investigate the use of Sii by independent practitioners on their own projects

Information Seeking from Web-Based Resources: Sensemaking Strategies and Implications for Interaction Design

The internet has made an enormous volume of information available, and there has been substantial research into how users look for information. However, there has been much less research about how they make sense of what they find, and how sensemaking is shaped by the tasks they are trying to achieve. This research addresses that gap, with empirical studies of sensemaking during web-based information tasks. Two main studies are presented, which aimed to expose the relationship between information seeking and information comprehension and use. The first study explored the actions of experienced information processors (in this case, doctoral students) as they undertook research-related web-based tasks related to their own work. The second study observed experienced users as they undertook an unfamiliar topic comprehension task. In both studies participants were encouraged to ‘think-aloud’ as they completed web-based tasks. Audio-recording was used in Study-1 with video-recording in Study-2. In addition to the task session, background questionnaires and sample interviews were applied. A detailed, iterative inductive analysis was undertaken for each study. The analysis produced a framework that models the users’ process in terms of five categories of information interactions: seeking, evaluating for selection, evaluating for use, compilation, and planning. A range of visual representations were developed to capture the user sessions, expressing facets such as how resources were used over time and in combination, and the sequences of user behaviours. Attention was given to the use of representation throughout this process. Sensemaking goals and strategies were inferred from users’ behaviours and utterances, and were related to their activity and output. The intertwined nature of information seeking and sensemaking activity was revealed, and planning (not addressed in previous literature) was identified as a significant behaviour that drives strategy and binds the other behaviours to the task-in-hand. These findings have implications for interaction design and for tools to support sensemaking

Approaches to evidence synthesis in international development : a research agenda

Abstract: This paper discusses the spectrum of synthesis methods available to generate, explore and text theory, their value to the field of international development and innovations required to make better use of the primary research available. It argues for clearer distinctions between syntheses produced as public goods, and those tailored to specific circumstances; and strengthening knowledge systems through greater use of maps to navigate existing and missing evidence, harmonised outcomes and measures, and advances in automation technologies. Improved methods and guidance are required for synthesising formative research and investigating contextual factors. Engaging stakeholders and working across academic disciplines support the production of policy‐relevant syntheses and inspire methods development

Human-Computer Interaction

In this book the reader will find a collection of 31 papers presenting different facets of Human Computer Interaction, the result of research projects and experiments as well as new approaches to design user interfaces. The book is organized according to the following main topics in a sequential order: new interaction paradigms, multimodality, usability studies on several interaction mechanisms, human factors, universal design and development methodologies and tools

Developing Learning Analytics for Epistemic Commitments in a Collaborative Information Seeking Environment

Learning analytics sits at the confluence of learning, information, and computer sciences. Using a distinctive account of learning analytics as a form of assessment, I first argue for its potential in pedagogically motivated learning design, suggesting a particular construct – epistemic cognition in literacy contexts – to probe using learning analytics. I argue for a recasting of epistemic cognition as ‘epistemic commitments’ in collaborative information tasks drawing a novel alignment between information seeking and multiple document processing (MDP) models, with empirical and theoretical grounding given for a focus on collaboration and dialogue in such activities. Thus, epistemic commitments are seen in the ways students seek, select, and integrate claims from multiple sources, and the ways in which their collaborative dialogue is brought to bear in this activity. Accordingly, the empirical element of the thesis develops two pedagogically grounded literacy based tasks: a MDP task, in which pre-selected documents were provided to students; and a collaborative information seeking task (CIS), in which students could search the web. These tasks were deployed at scale (n > 500) and involved writing an evaluative review, followed by a pedagogically supported peer assessment task. Assessment outcomes were analysed in the context of a new epistemic commitments-oriented set of trace data, and psychometric data regarding the participants’ epistemic cognition. Demonstrating the value of the methodological and conceptual approach taken, qualitative analyses indicate clear epistemic activity, and stark differences in behaviour between groups, the complexity of which is challenging to model computationally. Despite this complexity, quantitative analyses indicate that up to 30% of variance in output scores can be modelled using behavioural indicators. The explanatory potential of behaviourally-oriented models of epistemic commitments grounded in tool-interaction and collaborative dialogue is demonstrated. The thesis provides an exemplification of theoretically positioned analytic development, drawing on interdisciplinary literatures in addressing complex learning contexts

Evaluating collaborative information-seeking interfaces with a search-oriented inspection method and re-framed information seeking theory

Despite the many implicit references to the social contexts of search within information seeking and retrieval research, there has been relatively little work that has specifically investigated the additional requirements for collaborative information-seeking interfaces. Here, we re-assess an existing analytical inspection framework, designed for individual information seeking, and then apply it to evaluate a recent collaborative information-seeking interface: SearchTogether. The framework was built upon two models of solitary information seeking, and so as part of the re-assessment we first re-frame the models for collaborative contexts. We re-frame a model of search tactics, providing revised definitions that consider known collaborators. We then re-frame a model of searcher profiles to analyse support for different group dynamics. After presenting an analysis of SearchTogether, we reflect on its accuracy, showing that the framework identified eight known truths, eight new insights, and no known-to-be-untrue insights into the design. We conclude that the framework: (a) can still be applied to collaborative information-seeking interfaces; (b) can successfully produce additional requirements for collaborative information-seeking interfaces; and (c) can successfully model different dynamics of collaborating searchers.</p