Usability evaluation of digital libraries: a tutorial

This one-day tutorial is an introduction to usability evaluation for Digital Libraries. In particular, we will introduce Claims Analysis. This approach focuses on the designers’ motivations and reasons for making particular design decisions and examines the effect on the user’s interaction with the system. The general approach, as presented by Carroll and Rosson(1992), has been tailored specifically to the design of digital libraries. Digital libraries are notoriously difficult to design well in terms of their eventual usability. In this tutorial, we will present an overview of usability issues and techniques for digital libraries, and a more detailed account of claims analysis, including two supporting techniques – simple cognitive analysis based on Norman’s ‘action cycle’ and Scenarios and personas. Through a graduated series of worked examples, participants will get hands-on experience of applying this approach to developing more usable digital libraries. This tutorial assumes no prior knowledge of usability evaluation, and is aimed at all those involved in the development and deployment of digital libraries

Common ground in collaborative intelligence analysis: an empirical study

This paper reports an empirical exploration of how different configurations of collaboration technology affect peoples’ ability to construct and maintain common ground while conducting collaborative intelligence analysis work. Prior studies of collaboration technology have typically focused on simpler conversational tasks, or ones that involve physical manipulation, rather than the complex sensemaking and inference involved in intelligence work. The study explores the effects of video communication and shared visual workspace (SVW) on the negotiation of common ground by distributed teams collaborating in real time on intelligence analysis tasks. The experimental study uses a 2x2 factorial, between-subjects design involving two independent variables: presence or absence of Video and SVW. Two-member teams were randomly assigned to one of the four experimental media conditions and worked to complete several intelligence analysis tasks involving multiple, complex intelligence artefacts. Teams with access to the shared visual workspace could view their teammates’ eWhiteboards. Our results demonstrate a significant effect for the shared visual workspace: the effort of conversational grounding is reduced in the cases where SVW is available. However, there were no main effects for video and no interaction between the two variables. Also, we found that the “conversational grounding effort” required tended to decrease over the course of the tas

Stochastic Dominance in Mobility Analysis

This paper introduces a technique for mobility dominance and compares the degree of earnings mobility of men in the USA from 1970 to 1995. The highest mobility is found in the 1975–1980 or 1980–1985 periods

Conversational spaces for learning and designing

In this paper we describe a project to trial and evaluate ‘information spaces’ in which learners are more freely able to engage in the kinds of conversations that are beneficial to the practice of design and its education

A resources model for distributed sensemaking

In the field of Naturalistic Decision Making, the Data-Frame Model (DFM) has proven to be a popular and useful way of thinking about sensemaking. DFM provides a parsimonious account of how ‘sensemakers’ interact with the data in their environment in order to make sense of what is happening. In this paper, however, we argue that it is useful to elaborate DFM in several ways. We begin by arguing for the idea of sensemaking as a quest for coherence, an idea that we see as entirely consistent with the DFM. We then present some examples of sensemaking studies and use these to motivate a Distributed Resources Model of Sensemaking. This model uses the notion of resources for action, as resources that can be flexibly drawn upon in both choosing courses of action and accounting for the actions of oneself and of others (as opposed to prescriptions or mechanisms that determine behaviour in any strict way). It describes resources involved in sensemaking in terms of three domains: Knowledge and Beliefs, Values and Goals, and Action. Knowledge and beliefs are concerned with how things are; Values and Goals are concerned with how things are desired to be; and Action provides the means for redressing the gap. Central to the model is the idea that these resources can be distributed across a cognitive work system across actors and representational media. Hence, it aims to provide a framework for analysing sensemaking as Distributed Cognition

Developing a model of distributed sensemaking: a case study of military analysis

In this paper, we examine the role of representational artefacts in sensemaking. Embodied within representational media, such as maps, charts and lists, are a number of affordances, which can furnish sensemakers with the ability to perform tasks that may be difficult to do inside the head. Presented here is a study of sensemaking in action. We conducted a study of military intelligence analysts carrying out a training exercise, the analysis of which focuses on the use of external task-specific representations. We present a discussion of the findings of our study in the form of a model of distributed sensemaking. Our model concentrates on the interaction of information and various representational artefacts, leading to the generation of insights and a situation picture. We also introduce a number of levels of description for examining the properties and affordances offered by representational artefacts and their role in the sensemaking process

Editorial: In use, in situ: extending field research methods

A case for evaluating in use and in-situ Many authors have argued the need for a broader understanding of context and the situatedness of activity when approaching the evaluation of systems. However, prevailing practice often still tends towards attempting to understand the use of designed artefacts by focusing on a core set of tasks that are thought to define the system. A consequence of such focus is that other tasks are considered peripheral and outside the scope of design and evaluation activities. To illustrate the point, consider the experience, familiar to many of us, of being involved in an evaluation activity where participants provide unstructured qualitative feedback. Irrespective of whether the activity is carried out in a laboratory, in a high fidelity simulation or in a naturalistic setting, participants will frequently volunteer unsolicited feedback about tasks and goals that were not originally within the ambit of the design activity. This unprompted feedback, we suggest, is a cue for the evaluators to pay attention to the relationship between the tool and the practice in which it will be used. In other words a cue to consider the situations in which artefact will be used, the tasks and activities that may be affected by the new system, and so on. These are empirical questions that cannot be answered a priori by the development team, whether the evaluation is taking place in “artificial” or “natural” setting

Effect of Neutrino Heating on Primordial Nucleosynthesis

We have modified the standard code for primordial nucleosynthesis to include the effect of the slight heating of neutrinos by $e^\pm$ annihilations. There is a small, systematic change in the $^4$He yield, $\Delta Y \simeq +1.5\times 10^{-4}$, which is insensitive to the value of the baryon-to-photon ratio $\eta$ for 10^{-10}\la \eta \la 10^{-9}. We also find that the baryon-to-photon ratio decreases by about 0.5\% less than the canonical factor of 4/11 because some of the entropy in $e^\pm$ pairs is transferred to neutrinos. These results are in accord with recent analytical estimates.Comment: 14 pages/4 Figs (upon request

An approach to human-machine teaming in legal investigations using anchored narrative visualisation and machine learning

During legal investigations, analysts typically create external representations of an investigated domain as resource for cognitive offloading, reflection and collaboration. For investigations involving very large numbers of documents as evidence, creating such representations can be slow and costly, but essential. We believe that software tools, including interactive visualisation and machine learning, can be transformative in this arena, but that design must be predicated on an understanding of how such tools might support and enhance investigator cognition and team-based collaboration. In this paper, we propose an approach to this problem by: (a) allowing users to visually externalise their evolving mental models of an investigation domain in the form of thematically organized Anchored Narratives; and (b) using such narratives as a (more or less) tacit interface to cooperative, mixed initiative machine learning. We elaborate our approach through a discussion of representational forms significant to legal investigations and discuss the idea of linking such representations to machine learning