Reviewing and extending the five-user assumption: A grounded procedure for interaction evaluation

" © ACM, 2013. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Computer-Human Interaction (TOCHI), {VOL 20, ISS 5, (November 2013)} http://doi.acm.org/10.1145/2506210 "The debate concerning how many participants represents a sufficient number for interaction testing is well-established and long-running, with prominent contributions arguing that five users provide a good benchmark when seeking to discover interaction problems. We argue that adoption of five users in this context is often done with little understanding of the basis for, or implications of, the decision. We present an analysis of relevant research to clarify the meaning of the five-user assumption and to examine the way in which the original research that suggested it has been applied. This includes its blind adoption and application in some studies, and complaints about its inadequacies in others. We argue that the five-user assumption is often misunderstood, not only in the field of Human-Computer Interaction, but also in fields such as medical device design, or in business and information applications. The analysis that we present allows us to define a systematic approach for monitoring the sample discovery likelihood, in formative and summative evaluations, and for gathering information in order to make critical decisions during the interaction testing, while respecting the aim of the evaluation and allotted budget. This approach – which we call the ‘Grounded Procedure’ – is introduced and its value argued.The MATCH programme (EPSRC Grants: EP/F063822/1 EP/G012393/1

Finding and Resolving Security Misusability with Misusability Cases

Although widely used for both security and usability concerns, scenarios used in security design may not necessarily inform the design of usability, and vice- versa. One way of using scenarios to bridge security and usability involves explicitly describing how design deci- sions can lead to users inadvertently exploiting vulnera- bilities to carry out their production tasks. This paper describes how misusability cases, scenarios that describe how design decisions may lead to usability problems sub- sequently leading to system misuse, address this problem. We describe the related work upon which misusability cases are based before presenting the approach, and illus- trating its application using a case study example. Finally, we describe some findings from this approach that further inform the design of usable and secure systems

Just design

Inclusive design prescribes addressing the needs of the widest possible audience in order to consider human differences. Taking differences seriously, however, may imply severely restricting “the widest possible audience”. In confronting this paradox, we investigate to what extent Rawls’ theory of justice as fairness applies to design. By converting the paradox into the question of how design can be fair, we show that the demand for equitability shifts from the design output to the design process. We conclude that the two main questions about justice find application in design: the question about the standards of justice and the question about its metrics. We endorse a Rawlsian approach to the former, while some revision may be due regarding the latter

Analysis of Usability Cost-Benefit Models

There are few development organizations that have fully integrated usability activities as an integral part of their product development projects. One reason for this is that the benefits of better usability are not visible for the management. In this paper, the characteristics of selected published usability cost-benefit models are analyzed. These models have different approaches for identifying, approaching and categorizing the costs and benefits of usability. The analyzed models provide general guidelines for estimating the costs and benefits of usability but in most cases provide only little details. It is proposed that the business type of development organization and the type of the developed product as variables could be taken into account when analyzing the benefits of better usability

Views from within a narrative : Evaluating long-term human-robot interaction in a naturalistic environment using open-ended scenarios

Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited. Date of acceptance: 16/06/2014This article describes the prototyping of human–robot interactions in the University of Hertfordshire (UH) Robot House. Twelve participants took part in a long-term study in which they interacted with robots in the UH Robot House once a week for a period of 10 weeks. A prototyping method using the narrative framing technique allowed participants to engage with the robots in episodic interactions that were framed using narrative to convey the impression of a continuous long-term interaction. The goal was to examine how participants responded to the scenarios and the robots as well as specific robot behaviours, such as agent migration and expressive behaviours. Evaluation of the robots and the scenarios were elicited using several measures, including the standardised System Usability Scale, an ad hoc Scenario Acceptance Scale, as well as single-item Likert scales, open-ended questionnaire items and a debriefing interview. Results suggest that participants felt that the use of this prototyping technique allowed them insight into the use of the robot, and that they accepted the use of the robot within the scenarioPeer reviewe

Modeling of system knowledge for efficient agile manufacturing : tool evaluation, selection and implementation scenario in SMEs

In the manufacturing world, knowledge is fundamental in order to achieve effective and efficient real time decision making. In order to make manufacturing system knowledge available to the decision maker it has to be first captured and then modelled. Therefore tools that provide a suitable means for capturing and representation of manufacturing system knowledge are required in several types of industrial sectors and types of company’s (large, SME). A literature review about best practice for capturing requirements for simulation development and system knowledge modeling has been conducted. The aim of this study was to select the best tool for manufacturing system knowledge modelling in an open-source environment. In order to select this tool, different criteria were selected, based on which several tools were analyzed and rated. An exemplary use case was then developed using the selected tool, Systems Modeling Language (SysML). Therefore, the best practice has been studied, evaluated, selected and then applied to two industrial use cases by the use of a selected opens source tool.peer-reviewe

Stakeholder involvement, motivation, responsibility, communication: How to design usable security in e-Science

e-Science projects face a difficult challenge in providing access to valuable computational resources, data and software to large communities of distributed users. Oil the one hand, the raison d'etre of the projects is to encourage members of their research communities to use the resources provided. Oil the other hand, the threats to these resources from online attacks require robust and effective Security to mitigate the risks faced. This raises two issues: ensuring that (I) the security mechanisms put in place are usable by the different users of the system, and (2) the security of the overall system satisfies the security needs of all its different stakeholders. A failure to address either of these issues call seriously jeopardise the success of e-Science projects.The aim of this paper is to firstly provide a detailed understanding of how these challenges call present themselves in practice in the development of e-Science applications. Secondly, this paper examines the steps that projects can undertake to ensure that security requirements are correctly identified, and security measures are usable by the intended research community. The research presented in this paper is based Oil four case studies of c-Science projects. Security design traditionally uses expert analysis of risks to the technology and deploys appropriate countermeasures to deal with them. However, these case studies highlight the importance of involving all stakeholders in the process of identifying security needs and designing secure and usable systems.For each case study, transcripts of the security analysis and design sessions were analysed to gain insight into the issues and factors that surround the design of usable security. The analysis concludes with a model explaining the relationships between the most important factors identified. This includes a detailed examination of the roles of responsibility, motivation and communication of stakeholders in the ongoing process of designing usable secure socio-technical systems such as e-Science. (C) 2007 Elsevier Ltd. All rights reserved