Empirical development of a heuristic evaluation methodology for shared workspace groupware

Good real time groupware products are hard to develop, in part because evaluating their support for basic teamwork activities is difficult and costly. To address this problem, we are developing discount evaluation methods that look for groupware-specific usability problems. In a previous paper, we detailed a new set of usability heuristics that evaluators can use to inspect shared workspace groupware to see how they support teamwork. We wanted to determine whether the new heuristics could be integrated into a low-cost methodology that parallels Nielsen’s traditional heuristic evaluation (HE). To this end, we examined 27 evaluations of two shared workspace groupware systems and analysed the inspectors ’ relative performance and variability. Similar to Nielsen’s findings for traditional HE, individual inspectors discovered about a fifth of the total known teamwork problems, and that there was only modest overlap in the problems they found. Groups of three to five inspectors would report about 40– 60 % of the total known teamwork problems. These results suggest that heuristic evaluation using our groupware heuristics can be an effective and efficient method for identifying teamwork problems in shared workspace groupware systems

Analytical Evaluation of Groupware Usability in Concerted Work Scenarios

This paper addresses the usability evaluation of groupware tools supporting teams working in an intensive concerted effort towards a shared goal. Considering the evaluation of this technology in such scenario is a complex and costly endeavor, we propose a discount analytical evaluation approach. Our approach uses available models of human performance to estimate groupware usability. The paper illustrates the use of the Keystroke-Level Model for analyzing a concerted work scenario and shows how the adopted evaluation method affords analytical experiments with alternative groupware designs. This paper contributes to understand the fine-grained details involved in groupware desig

Quantitative Analysis of Shared Workspace Usability

In this paper we propose a discount analytical approach for quantitatively evaluating shared workspace usability. We started with existing models of human performance, developed in the Human-Computer Interaction field and thus focused on single user interactions, and studied the benefits of extending them to collaborative scenarios. The obtained results indicate that the proposed approach: (1) facilitates the fine-grained analysis of intensive concerted work scenarios; (2) provides quantitative estimates of collaborative actions performed in shared workspaces; and (3) affords comparing alternative design decisions, using shared workspace usability metrics derived from the aforementioned quantitative estimate

A Method for Evaluating End-User Development Technologies

End-user development (EUD) is a strategy that can reduce a considerable amount of business demand on IT departments. Empowering the end-user in the context of software development is only possible through technologies that allow them to manipulate data and information without the need for deep programming knowledge. The successful selection of appropriate tools and technologies is highly dependent on the context in which the end-user is embedded. End-users should be a central piece in any software package evaluation, being key in the evaluation process in the end-user development context. However, little research has empirically examined software package evaluation criteria and techniques in general, and in the end-user development context in particular. This paper aims to provide a method for technology evaluation in the context of end-user development and to present the evaluation of two platforms. We conclude our study proposing a set of suggestions for future research

Using natural user interfaces to support synchronous distributed collaborative work

Synchronous Distributed Collaborative Work (SDCW) occurs when group members work together at the same time from different places together to achieve a common goal. Effective SDCW requires good communication, continuous coordination and shared information among group members. SDCW is possible because of groupware, a class of computer software systems that supports group work. Shared-workspace groupware systems are systems that provide a common workspace that aims to replicate aspects of a physical workspace that is shared among group members in a co-located environment. Shared-workspace groupware systems have failed to provide the same degree of coordination and awareness among distributed group members that exists in co-located groups owing to unintuitive interaction techniques that these systems have incorporated. Natural User Interfaces (NUIs) focus on reusing natural human abilities such as touch, speech, gestures and proximity awareness to allow intuitive human-computer interaction. These interaction techniques could provide solutions to the existing issues of groupware systems by breaking down the barrier between people and technology created by the interaction techniques currently utilised. The aim of this research was to investigate how NUI interaction techniques could be used to effectively support SDCW. An architecture for such a shared-workspace groupware system was proposed and a prototype, called GroupAware, was designed and developed based on this architecture. GroupAware allows multiple users from distributed locations to simultaneously view and annotate text documents, and create graphic designs in a shared workspace. Documents are represented as visual objects that can be manipulated through touch gestures. Group coordination and awareness is maintained through document updates via immediate workspace synchronization, user action tracking via user labels and user availability identification via basic proxemic interaction. Members can effectively communicate via audio and video conferencing. A user study was conducted to evaluate GroupAware and determine whether NUI interaction techniques effectively supported SDCW. Ten groups of three members each participated in the study. High levels of performance, user satisfaction and collaboration demonstrated that GroupAware was an effective groupware system that was easy to learn and use, and effectively supported group work in terms of communication, coordination and information sharing. Participants gave highly positive comments about the system that further supported the results. The successful implementation of GroupAware and the positive results obtained from the user evaluation provides evidence that NUI interaction techniques can effectively support SDCW

Establishing usability heuristics for heuristics evaluation in a specific domain: is there a consensus?

Heuristics evaluation is frequently employed to evaluate usability. While general heuristics are suitable to evaluate most user interfaces, there is still a need to establish heuristics for specific domains to ensure that their specific usability issues are identified. This paper presents a comprehensive review of 70 studies related to usability heuristics for specific domains. The aim of this paper is to review the processes that were applied to establish heuristics in specific domains and identify gaps in order to provide recommendations for future research and area of improvements. The most urgent issue found is the deficiency of validation effort following heuristics proposition and the lack of robustness and rigour of validation method adopted. Whether domain specific heuristics perform better or worse than general ones is inconclusive due to lack of validation quality and clarity on how to assess the effectiveness of heuristics for specific domains. The lack of validation quality also affects effort in improving existing heuristics for specific domain as their weaknesses are not addressed