Evaluating system utility and conceptual fit using CASSM

There is a wealth of user-centred evaluation methods (UEMs) to support the analyst in assessing interactive systems. Many of these support detailed aspects of use – for example: Is the feedback helpful? Are labels appropriate? Is the task structure optimal? Few UEMs encourage the analyst to step back and consider how well a system supports users’ conceptual understandings and system utility. In this paper, we present CASSM, a method which focuses on the quality of ‘fit’ between users and an interactive system. We describe the methodology of conducting a CASSM analysis and illustrate the approach with three contrasting worked examples (a robotic arm, a digital library system and a drawing tool) that demonstrate different depths of analysis. We show how CASSM can help identify re-design possibilities to improve system utility. CASSM complements established evaluation methods by focusing on conceptual structures rather than procedures. Prototype tool support for completing a CASSM analysis is provided by Cassata, an open source development

Scoping analytical usability evaluation methods: A case study

Analytical usability evaluation methods (UEMs) can complement empirical evaluation of systems: for example, they can often be used earlier in design and can provide accounts of why users might experience difficulties, as well as what those difficulties are. However, their properties and value are only partially understood. One way to improve our understanding is by detailed comparisons using a single interface or system as a target for evaluation, but we need to look deeper than simple problem counts: we need to consider what kinds of accounts each UEM offers, and why. Here, we report on a detailed comparison of eight analytical UEMs. These eight methods were applied to it robotic arm interface, and the findings were systematically compared against video data of the arm ill use. The usability issues that were identified could be grouped into five categories: system design, user misconceptions, conceptual fit between user and system, physical issues, and contextual ones. Other possible categories such as User experience did not emerge in this particular study. With the exception of Heuristic Evaluation, which supported a range of insights, each analytical method was found to focus attention on just one or two categories of issues. Two of the three "home-grown" methods (Evaluating Multimodal Usability and Concept-based Analysis of Surface and Structural Misfits) were found to occupy particular niches in the space, whereas the third (Programmable User Modeling) did not. This approach has identified commonalities and contrasts between methods and provided accounts of why a particular method yielded the insights it did. Rather than considering measures such as problem count or thoroughness, this approach has yielded insights into the scope of each method

Secure webs and buying intention: the moderating role of usability

El presente trabajo ha planteado un modelo conceptual a fin de mostrar como los antecedentes de la intención de compra se ven reforzados en contextos de Webs altamente usables. Específicamente, el trabajo analiza en profundidad el rol moderador de la usabilidad en la explicación de la conexión entre seguridad de una Web e intención de compra. Entre ambos extremos (seguridad e intención de compra), se han incluido diversas variables para explicar mejor su conexión. Para ello, ha sido diseñada una Web ficticia de ropa dirigida al segmento joven de clase media. A fin de alterar la usabilidad de la Web se han realizado dos tipos de manipulaciones: la velocidad y la facilidad de uso de la Web. Las dos Webs creadas (alta usabilidad y baja usabilidad) fueron visitadas por un total de 170 encuestados que fueron compensados con un USB valorado en 15 euros. Los resultados muestran que la seguridad percibida en la Web acarrea tres interesantes efectos (especialmente para la Web altamente usable): (i) mejora las actitudes agrado, (ii) reduce el nivel de riesgo percibido; (iii) aumenta la confianza. Los dos últimos efectos, a su vez, acaban aumentando la intención de compra.. Por último, se ha demostrado que la usabilidad, efectivamente, refuerza las relaciones consideradas en el modelo propuesto para explicar la intención de compra.A conceptual model has been proposed to show how buying intention antecedents are reinforced in highly usable contexts. Specifically, this paper deeply analyses the moderator role of system variables (usability) on explaining the relationship between Web security and buying intention. Between both extremes (security and buying intention), several relationships have also been stated to better explain this effect. An “ideal” fictitious Website was designed for a non existent clothing company directed at the segment of middle class consumers. In order to alter Web usability, two blocks of changes were made, one concerning Website speed and the other related to ease of use. Our experiment sample consisted of 170 respondents who participated in exchange for a pen-drive (USB) valued at 15 euros. The results show that improving website security has three interesting effects (especially in high usable contexts): (i) it improves pleasure attitudes, (ii) reduces the level of perceived risk and (iii) increases trust. Secondly, it has been found that to increase buying intention, two actions must be taken: (i) to diminish perceived risk and (ii) to improve users’ pleasure attitudes towards the Website. Finally, usability has been found to have a moderating role in all the relationships considered (reinforcing them)

Systematic evaluation of design choices for software development tools

[Abstract]: Most design and evaluation of software tools is based on the intuition and experience of the designers. Software tool designers consider themselves typical users of the tools that they build and tend to subjectively evaluate their products rather than objectively evaluate them using established usability methods. This subjective approach is inadequate if the quality of software tools is to improve and the use of more systematic methods is advocated. This paper summarises a sequence of studies that show how user interface design choices for software development tools can be evaluated using established usability engineering techniques. The techniques used included guideline review, predictive modelling and experimental studies with users

Concept-based Analysis of Surface and Structural Misfits (CASSM) Tutorial notes

Concept-based Analysis of Surface and Structural Misfits (CASSM) in a novel approach to usability analysis that focuses attention on misfits between user and system concepts. We believe that as an approach it has several desirable qualities: o It focuses on concepts rather than tasks or procedures. Consequently, it complements the majority of existing approaches to usability evaluation. In particular, it analyses conceptual misfits between user and system. o By intentionally supporting ‘sketchy’ analysis, CASSM avoids the ‘death by detail’ that plagues many evaluation techniques. CASSM analyses do not have to be complete or consistent to be useful – though of course a thorough analysis is likely to have these properties. Also, CASSM analyses are often quite succinct, compared to (for example) a Cognitive Walkthrough (Wharton et al, 1994), Heuristic Evaluation (Nielsen, 1994) or GOMS analysis (John & Kieras, 1996). o As a notation, it provides a ‘bridge’ between the core ideas underpinning work on mental models and design issues, and may thus make prior work on mental models more readily accessible to design practice. [This should be regarded as a hypothesis that has not yet been tested.] o The CASSM notation provides a relatively formal definition of many of Green’s Cognitive Dimensions (see, for example, Green, 1989; Green & Petre, 1996; Blackwell & Green 2003). In this way, it further supports assessment of a system in terms of CDs. This is discussed in detail towards the end of this document. Although the name (CASSM: Concept-based Analysis of Surface and Structural Misfits) emphasises the importance of misfits, you should be aware that there are other kinds of user–system misfits that are outside the scope of CASSM; for example, inconsistencies in procedures for similar tasks would be picked up by other techniques but are not directly addressed within CASSM. CASSM focuses on conceptual structures

Children searching information on the Internet: Performance on children's interfaces compared to Google

Children frequently make use of the Internet to search for information. However, research shows that children experience many problems with searching and browsing the web. The last decade numerous search environments have been developed, especially for children. Do these search interfaces support children in effective information-seeking? And do these interfaces add value to today’s popular search engines, such as Google? In this explorative study, we compared children’s search performance on four interfaces designed for children, with their performance on Google. We found that the children did not perform better on these interfaces than on Google. This study also uncovered several problems that children experienced with these search interfaces, which can be of use for designers of future search interfaces for children

Deuce: A Lightweight User Interface for Structured Editing

We present a structure-aware code editor, called Deuce, that is equipped with direct manipulation capabilities for invoking automated program transformations. Compared to traditional refactoring environments, Deuce employs a direct manipulation interface that is tightly integrated within a text-based editing workflow. In particular, Deuce draws (i) clickable widgets atop the source code that allow the user to structurally select the unstructured text for subexpressions and other relevant features, and (ii) a lightweight, interactive menu of potential transformations based on the current selections. We implement and evaluate our design with mostly standard transformations in the context of a small functional programming language. A controlled user study with 21 participants demonstrates that structural selection is preferred to a more traditional text-selection interface and may be faster overall once users gain experience with the tool. These results accord with Deuce's aim to provide human-friendly structural interactions on top of familiar text-based editing.Comment: ICSE 2018 Paper + Supplementary Appendice

Exploration of location-based services adoption

As mobile technologies become more ubiquitous in the general population, it is reasonable to assume that individuals will consume services and software to enhance their aspirations and entertainment desires. This paper discusses a controlled experiment to explore aspects of user perceptions of their use of location-based services. This study examines a location-based service prototype experiment and analysis based on the UTAUT model. The results show significant indicators that suggest behavior patterns of early adopters of location-based services are being observed. We discuss these influences and attempt to explain their significance. Moreover, more curiously we discuss why some of our model was unsupported and postulate why

Structured computer-based training in the interpretation of neuroradiological images

Computer-based systems may be able to address a recognised need throughout the medical profession for a more structured approach to training. We describe a combined training system for neuroradiology, the MR Tutor that differs from previous approaches to computer-assisted training in radiology in that it provides case-based tuition whereby the system and user communicate in terms of a well-founded Image Description Language. The system implements a novel method of visualisation and interaction with a library of fully described cases utilising statistical models of similarity, typicality and disease categorisation of cases. We describe the rationale, knowledge representation and design of the system, and provide a formative evaluation of its usability and effectiveness