Measuring and comparing the reliability of the structured walkthrough evaluation method with novices and experts

Effective evaluation of websites for accessibility remains problematic. Automated evaluation tools still require a significant manual element. There is also a significant expertise and evaluator effect. The Structured Walkthrough method is the translation of a manual, expert accessibility evaluation process adapted for use by novices. The method is embedded in the Accessibility Evaluation Assistant (AEA), a web accessibility knowledge management tool. Previous trials examined the pedagogical potential of the tool when incorporated into an undergraduate computing curriculum. The results of the evaluations carried out by novices yielded promising, consistent levels of validity and reliability. This paper presents the results of an empirical study that compares the reliability of accessibility evaluations produced by two groups (novices and experts). The main results of this study indicate that overall reliability of expert evaluations was 76% compared to 65% for evaluations produced by novices. The potential of the Structured Walkthrough method as a useful and viable tool for expert evaluators is also examined. Copyright 2014 ACM

A practical review of energy saving technology for ageing populations

Fuel poverty is a critical issue for a globally ageing population. Longer heating/cooling requirements combine with declining incomes to create a problem in need of urgent attention. One solution is to deploy technology to help elderly users feel informed about their energy use, and empowered to take steps to make it more cost effective and efficient. This study subjects a broad cross section of energy monitoring and home automation products to a formal ergonomic analysis. A high level task analysis was used to guide a product walk through, and a toolkit approach was used thereafter to drive out further insights. The findings reveal a number of serious usability issues which prevent these products from successfully accessing an important target demographic and associated energy saving and fuel poverty outcomes. Design principles and examples are distilled from the research to enable practitioners to translate the underlying research into high quality design-engineering solutions

Evaluating system utility and conceptual fit using CASSM

There is a wealth of user-centred evaluation methods (UEMs) to support the analyst in assessing interactive systems. Many of these support detailed aspects of use – for example: Is the feedback helpful? Are labels appropriate? Is the task structure optimal? Few UEMs encourage the analyst to step back and consider how well a system supports users’ conceptual understandings and system utility. In this paper, we present CASSM, a method which focuses on the quality of ‘fit’ between users and an interactive system. We describe the methodology of conducting a CASSM analysis and illustrate the approach with three contrasting worked examples (a robotic arm, a digital library system and a drawing tool) that demonstrate different depths of analysis. We show how CASSM can help identify re-design possibilities to improve system utility. CASSM complements established evaluation methods by focusing on conceptual structures rather than procedures. Prototype tool support for completing a CASSM analysis is provided by Cassata, an open source development

Records management capacity and compliance toolkits : a critical assessment.

This article seeks to present the results of a project that critically evaluated a series of toolkits for assessing records management capacity and/or compliance. These toolkits have been developed in different countries and sectors within the context of the e-environment and provide evidence of good corporate and information governance. Design/methodology/approach - A desk-based investigation of the tools was followed by an electronic Delphi with toolkit developers and performance measurement experts to develop a set of evaluation criteria. Different stakeholders then evaluated the toolkits against the criteria using cognitive walkthroughs and expert heuristic reviews. The results and the research process were reviewed via electronic discussion. Findings - Developed by recognised and highly respected organisations, three of the toolkits are software tools, whilst the fourth is a methodology. They are all underpinned by relevant national/international records management legislation, standards and good practice including, either implicitly or explicitly, ISO 15489. They all have strengths, complementing rather than competing with one another. They enable the involvement of other staff, thereby providing an opportunity for raising awareness of the importance of effective records management. Practical implications - These toolkits are potentially very powerful, flexible and of real value to organisations in managing their records. They can be used for a "quick and dirty" assessment of records management capacity or compliance as well as in-depth analysis. The most important criterion for selecting the appropriate one is to match the toolkit with the scenario. Originality/value - This paper aims to raise awareness of the range and nature of records management toolkits and their potential for varied use in practice to support more effective management of records

The patterning of finance/security : a designerly walkthrough of challenger banking apps

Culture is being ‘appified’. Diverse, pre-existing everyday activities are being redesigned so they happen with and through apps. While apps are often encountered as equivalent icons in apps stores or digital devices, the processes of appification – that is, the actions required to turn something into an app – vary significantly. In this article, we offer a comparative analysis of a number of ‘challenger’ banking apps in the United Kingdom. As a retail service, banking is highly regulated and banks must take steps to identify and verify their customers before entering a retail relationship. Once established, this ‘secured’ financial identity underpins a lot of everyday economic activity. Adopting the method of the walkthrough analysis, we study the specific ways these processes of identifying and verifying the identity of the customer (now the user) occur through user onboarding. We argue that banking apps provide a unique way of binding the user to an identity, one that combines the affordances of smart phones with the techniques, knowledge and patterns of user experience design. With the appification of banking, we see new processes of security folded into the everyday experience of apps. Our analysis shows how these binding identities are achieved through what we refer to as the patterning of finance/security. This patterning is significant, moreover, given its availability for wider circulation beyond the context of retail banking apps

Categorisation of visualisation methods to support the design of Human-Computer Interaction systems

During the design of Human-Computer Interaction (HCI) systems, the creation of visual artefacts forms an important part of design. On one hand producing a visual artefact has a number of advantages: it helps designers to externalise their thought and acts as a common language between different stakeholders. On the other hand, if an inappropriate visualisation method is employed it could hinder the design process. To support the design of HCI systems, this paper reviews the categorisation of visualisation methods used in HCI. A keyword search is conducted to identify a) current HCI design methods, b) approaches of selecting these methods. The resulting design methods are filtered to create a list of just visualisation methods. These are then categorised using the approaches identified in (b). As a result 23 HCI visualisation methods are identified and categorised in 5 selection approaches (The Recipient, Primary Purpose, Visual Archetype, Interaction Type, and The Design Process).Innovate UK, EPSRC, Airbus Group Innovation