Tycho: Facilitation Support for Groupware User Tests

Running user tests for groupware requires tracking the progress of multiple users performing co-dependent tasks, while facilitating and observing their coordinated actions. This requirement negatively impacts the main objective of user testing, which is detecting usability flaws. User testing of groupware becomes more challenging when run remotely. Even if there are tools for remote user testing that can get the job done for a single user, they are not prepared for collaborative scenarios. In this paper we argue that tool support for the facilitation of user tests of groupware is missing. Consequently, we propose a method for user tests with tool assistance that makes it possible to automate task synchronization, especially for different workflows that must be run concurrently. We evaluated our proposal by comparing it to a manually facilitated approach during on-site coordinated user tests. The results indicate that, while the task of designing and running user tests with tool support takes more time, it allows the testers to better focus on detecting usability problems.Fil: Grigera, Julián. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - La Plata; Argentina. Universidad Nacional de La Plata. Facultad de Informática. Laboratorio de Investigación y Formación en Informática Avanzada; Argentina. Provincia de Buenos Aires. Gobernación. Comisión de Investigaciones Científicas; ArgentinaFil: Gardey, Juan Cruz. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - La Plata; Argentina. Universidad Nacional de La Plata. Facultad de Informática. Laboratorio de Investigación y Formación en Informática Avanzada; ArgentinaFil: Fernandez, Alejandro. Universidad Nacional de La Plata. Facultad de Informática. Laboratorio de Investigación y Formación en Informática Avanzada; Argentina. Provincia de Buenos Aires. Gobernación. Comisión de Investigaciones Científicas; ArgentinaFil: Garrido, Alejandra. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - La Plata; Argentina. Universidad Nacional de La Plata. Facultad de Informática. Laboratorio de Investigación y Formación en Informática Avanzada; Argentin

Empirical development of a heuristic evaluation methodology for shared workspace groupware

Good real time groupware products are hard to develop, in part because evaluating their support for basic teamwork activities is difficult and costly. To address this problem, we are developing discount evaluation methods that look for groupware-specific usability problems. In a previous paper, we detailed a new set of usability heuristics that evaluators can use to inspect shared workspace groupware to see how they support teamwork. We wanted to determine whether the new heuristics could be integrated into a low-cost methodology that parallels Nielsen’s traditional heuristic evaluation (HE). To this end, we examined 27 evaluations of two shared workspace groupware systems and analysed the inspectors ’ relative performance and variability. Similar to Nielsen’s findings for traditional HE, individual inspectors discovered about a fifth of the total known teamwork problems, and that there was only modest overlap in the problems they found. Groups of three to five inspectors would report about 40– 60 % of the total known teamwork problems. These results suggest that heuristic evaluation using our groupware heuristics can be an effective and efficient method for identifying teamwork problems in shared workspace groupware systems

Tycho: facilitation support for groupware user tests

Running user tests for groupware requires tracking the progress of multiple users performing co-dependent tasks, while facilitating and observing their coordinated actions. This requirement negatively impacts the main objective of user testing, which is detecting usability flaws. User testing of groupware becomes more challenging when run remotely. Even if there are tools for remote user testing that can get the job done for a single user, they are not prepared for collaborative scenarios. In this paper we argue that tool support for the facilitation of user tests of groupware is missing. Consequently, we propose a method for user tests with tool assistance that makes it possible to automate task synchronization, especially for different workflows that must be run concurrently. We evaluated our proposal by comparing it to a manually facilitated approach during on-site coordinated user tests. The results indicate that, while the task of designing and running user tests with tool support takes more time, it allows the testers to better focus on detecting usability problems

Assessing the Semiotic Inspection Method – The Evaluators’ Perspective

This paper presents an assessment of the Semiotic Inspection Methodaimed at understanding its costs, benefits, advantages and disadvantages from the evaluators’perspective. We applied a questionnaire to novice evaluators and interviewedthe authors of the method (representing the experts’ evaluators). An analysis of theresponses shows interesting insights and characteristics of the method