The programming-like-analysis of an innovative media tool

This paper describes a case study in which evaluation techniques have been developed and applied to a novel commercially developed tool for supporting efficiency and effectiveness of a digital film production processes. The tool is based upon a familiar concept in digital publishing that of separating style from content, and as such, it represents a challenge for intended end users since it moves them away from traditional working practices and towards programming-like-activity. Two alternative user interfaces have been developed following a commercial development route. Approaches to analyzing the effectiveness of the tool and its interfaces prior to its widespread adoption are described and the conclusions from this analysis are illustrated and discussed

Developing formative evaluation for complex interaction

© 2019 IADIS Press. All rights reserved. This paper reports on the development, refinement and use of a design instrument for supporting the exploration of interactive tool design. The distinctive focus of the instrument is that it is intended for interactive tools that go beyond the aim of enabling direct interaction. It is intended to tools that by necessity mediate access to rich complex functionality. We argue that the majority of evaluative instruments that HCI designers have at-hand are not particularly useful in this context. We proposed that alternative concepts are of more relevance, specifically the concepts developed in the Cognitive Dimensions Framework. We have developed an instrument to enable and support the critical assessment of alternative complex interaction designs that is motivated by these concepts. Having described the origins of the tool, the paper then describes how it has been applied in three real world design and development settings. The paper concludes with reflections upon how best to refine, encapsulate and further improve the instrument

Methodological development

Book description: Human-Computer Interaction draws on the fields of computer science, psychology, cognitive science, and organisational and social sciences in order to understand how people use and experience interactive technology. Until now, researchers have been forced to return to the individual subjects to learn about research methods and how to adapt them to the particular challenges of HCI. This is the first book to provide a single resource through which a range of commonly used research methods in HCI are introduced. Chapters are authored by internationally leading HCI researchers who use examples from their own work to illustrate how the methods apply in an HCI context. Each chapter also contains key references to help researchers find out more about each method as it has been used in HCI. Topics covered include experimental design, use of eyetracking, qualitative research methods, cognitive modelling, how to develop new methodologies and writing up your research

Using and utilizing an innovative media development tool

This paper describes a novel commercially developed tool for supporting efficiency and effectiveness of a digital film production processes. The tool is described as are two alternative user interfaces for it. Approaches to analyzing the effectiveness of the tool prior to its widespread adoption are described and the conclusions from this analysis are illustrated

Evaluating system utility and conceptual fit using CASSM

There is a wealth of user-centred evaluation methods (UEMs) to support the analyst in assessing interactive systems. Many of these support detailed aspects of use – for example: Is the feedback helpful? Are labels appropriate? Is the task structure optimal? Few UEMs encourage the analyst to step back and consider how well a system supports users’ conceptual understandings and system utility. In this paper, we present CASSM, a method which focuses on the quality of ‘fit’ between users and an interactive system. We describe the methodology of conducting a CASSM analysis and illustrate the approach with three contrasting worked examples (a robotic arm, a digital library system and a drawing tool) that demonstrate different depths of analysis. We show how CASSM can help identify re-design possibilities to improve system utility. CASSM complements established evaluation methods by focusing on conceptual structures rather than procedures. Prototype tool support for completing a CASSM analysis is provided by Cassata, an open source development

A usability approach to improving the user experience in web directories

PhDWeb directories are hierarchically organised website collections that offer users subjectbased access to the Web. They played a significant part in navigating the Web in the past but their role has been weakened in recent years due to their cumbersome expanding collections. This thesis presents a unified framework combining the advantages of personalisation and redefined directory search for improving the usability of Web directories. The thesis begins with an examination of classification schemes that identifies the rigidity of hierarchical classifications and their suitability for Web directories in contrast to faceted classifications. This leads on to an Ontological Sketch Modelling (OSM) case study which identifies the misfits affecting user navigation in Web directories from known rigidity issues. The thesis continues with a review of personalisation techniques and a discussion of the user search model of Web directories following the suggested directions of improvement from the case study. A proposed user-centred framework to improve the usability of Web directories which consists of an individual content-based personalisation model and a redefined search model is then implemented as D-Persona and D-Search respectively. The remainder of the thesis is concerned with a usability test of D-Persona and D-Search aimed at discovering the efficiency, effectiveness and user satisfaction of the solution. This involves an experimental design, test results and discussions for the comparative user study. This thesis extracts a formal definition of the rigidity of hierarchies from their characteristics and justifies why hierarchies are still better suited than facets in organising Web directories. Second, it identifies misfits causing poor usability in Web directories based on the discovered rigidity of hierarchies. Third, it proposes a solution to tackle the misfits and improve the usability of Web directories which has been experimentally proved to be successful

A Usability Approach to Improving the User Experience in Web Directories

Submitted for the degree of Doctor of Philosophy, Queen Mary, University of Londo

Formalising an understanding of user–system misfits

Many of the difficulties users experience when working with interactive systems arise from misfits between the user’s conceptualisation of the domain and device with which they are working and the conceptualisation implemented within those systems. We report an analytical technique called CASSM (Concept-based Analysis for Surface and Structural Misfits) in which such misfits can be formally represented to assist in understanding, describing and reasoning about them. CASSM draws on the framework of Cognitive Dimensions (CDs) in which many types of misfit were classified and presented descriptively, with illustrative examples. CASSM allows precise definitions of many of the CDs, expressed in terms of entities, attributes, actions and relationships. These definitions have been implemented in Cassata, a tool for automated analysis of misfits, which we introduce and describe in some detail

Improving Tool Support for Personal Task Management (PTM)

Personal Task Management (PTM) describes the planning, prioritising and list-making of tasks employed by an individual user. There are hundreds of commercial electronic PTM tools available on the market which users can choose from. There appears to be little attempt to develop a framework for describing people’s task management behaviour, making it difficult to determine the extent to which these tools meet users’ needs. The aims of this thesis were therefore to understand how academics manage their tasks, to identify the conceptual gaps between them and the existing electronic tools, and to establish requirements for guiding the design and evaluation of PTM tools. The research adopts a user-centred design methodology. This includes both empirical and analytical approaches, conducted through four different studies. Firstly, a semi-structured interview study develops a PTM framework, describing the components of PTM (i.e. the underlying activities and contextual factors). Secondly, a member-checking study tests the accuracy of the framework. Thirdly, a video-diary study examines the inconsistencies discovered between the interview and member-checking studies. The findings extend the PTM framework to include other aspects of users (e.g. challenges, context awareness, etc.), broadening the understanding of the complexity of PTM behaviours. The data gathered in the user studies was analysed using a grounded theory (GT) approach, and the findings were then used to build personas of academics. Finally, an in-depth expert analytical evaluation of a set of existing tools using CASSM identifies the conceptual misfits between users and the existing tools. The contributions of this thesis are a development of the PTM framework, describing the key factors that influence academics in managing their tasks; a development of personas, explaining characteristics of different groups of academics and PTM strategies that they employ over time; and an evaluation of existing PTM tools, determining their strengths and limitations and providing recommendations