Usability Evaluation in Virtual Environments: Classification and Comparison of Methods

Virtual environments (VEs) are a relatively new type of human-computer interface in which users perceive and act in a three-dimensional world. The designers of such systems cannot rely solely on design guidelines for traditional two-dimensional interfaces, so usability evaluation is crucial for VEs. We present an overview of VE usability evaluation. First, we discuss some of the issues that differentiate VE usability evaluation from evaluation of traditional user interfaces such as GUIs. We also present a review of VE evaluation methods currently in use, and discuss a simple classification space for VE usability evaluation methods. This classification space provides a structured means for comparing evaluation methods according to three key characteristics: involvement of representative users, context of evaluation, and types of results produced. To illustrate these concepts, we compare two existing evaluation approaches: testbed evaluation [Bowman, Johnson, & Hodges, 1999], and sequential evaluation [Gabbard, Hix, & Swan, 1999]. We conclude by presenting novel ways to effectively link these two approaches to VE usability evaluation

The assessment of usability of electronic shopping: A heuristic evaluation

Today there are thousands of electronic shops accessible via the Web. Some provide user-friendly features whilst others seem not to consider usability factors at all. Yet, it is critical that the electronic shopping interface is user-friendly so as to help users to obtain their desired results. This study applied heuristic evaluation to examine the usability of current electronic shopping. In particular, it focused on four UK-based supermarkets offering electronic services: including ASDA, Iceland, Sainsbury, and Tesco. The evaluation consists of two stages: a free-flow inspection and a task-based inspection. The results indicate that the most significant and common usability problems have been found to lie within the areas of ‘User Control and Freedom’ and ‘Help and Documentation’. The findings of this study are applied to develop a set of usability guidelines to support the future design of effective interfaces for electronic shopping

Evaluation of the game development process of a location-based mobile game

There is a growing interest of government bodies and NGOs in using (serious) video games in awareness campaigns. Until now, however, little was known on how to set up such a campaign so as to effectively cater to the needs of different stakeholders including the target audience. Hence designing, developing and translating a game for educational purposes whilst balancing between fun and learning is a complex process, this paper aims to evaluate this by presenting a methodological framework for involving stakeholders in the design and development of a game-based awareness campaign based on a user-centered software design methodology and assesses its effectiveness in a concrete use case: the development of the location-based mobile game City Jam. The goal was to develop a game-based road safety campaign to confront adolescents with road traffic situations with the aim to positively influence road safety attitude and behavior. Mobile technologies offer new opportunities to embed digital game based learning by in different contexts. Given the nature of the road safety campaign, a location-based game format was chosen, aiming to facilitate learning by means of an extended three-way interaction (human interaction, game and context). Different user-centered design methods were deployed throughout several phases of the game development process: In phase one (the opportunity identification) a literature review was performed to investigate relevant fields for the game’s goal. In phase two (the game concept development) expert interviews and a focus groups were conducted with relevant stakeholders and in phase three (the game concept design) co-design sessions and a focus group resulted in a game design document. In phase four (game development and testing) the beta version of City jam was developed and tested in an iterative field testing design and resulted in the final game. Results obtained throughout the game development process provided us the opportunity to evaluate several major aspects. Firstly the impact of stakeholder involvement on the different phases of the design process and the final product resulted in a game that was tailored to the preferences and needs of the target group. Secondly translating the game concept into practice, such as game elements, proposed educational game elements, were evaluated based on the usability, playability principles and social and technological aspects. Benefits and challenges of user-centered design methods are discussed and how budget constraints and differing desired outcomes of different stakeholders challenge but also enrich the process

The ABCD of usability testing

We introduce a methodology for tracking and auditing feedback, errors and suggestions for software packages. This short paper describes how we innovate on the evaluation mechanism, introducing an (Antecedent, Barrier, Consequence and Development) ABCD form, embedded within an eParticipation platform to enable end users to easily report on any usability issues. This methodology will be utilised to improve the STEP cloud eParticipation platform (part of the current STEP Horizon2020 project http://step4youth.eu. The platform is currently being piloted in real life contexts, with the participation of public authorities that are integrating the eParticipation platform into their regular decision-making practices. The project is involving young people, through engagement and motivation strategies and giving them a voice in Environmental decision making at the local level. The pilot evaluation aims to demonstrate how open engagement needs to be embedded within public sector processes and the usability methodology reported here will help to identify the key barriers for wide scale deployment of the platform

The Effect of Web Usability on Users’ Web Experience

The ease with which a website visitor can find what they need is positively correlated with visitor satisfaction(Institute for Dynamic Educational Advancement, 2008). Web usability is a field that studies what factors affect the visitor’s ability to navigate through a website. Although there are publications outlining specific usability guidelines, many of them have little or no academic research to support the claim. HHS developed a list of 209 guidelines and rated each according to their strength of evidence (research-based support, 5 – high, 1 – low). Using heuristic evaluation and usability testing, this study provides additional research-based knowledge for those guidelines rated with a low strength of evidence. Results indicate that users desire printer-friendly webpages, require feedback on their location within the website, find linking to related content helpful, and expect a search option to be provided on every page. Additional research is necessary to determine if providing descriptive page titles or labeling pushbuttons clearly is important to web usability

From usability to secure computing and back again

Secure multi-party computation (MPC) allows multiple parties to jointly compute the output of a function while preserving the privacy of any individual party’s inputs to that function. As MPC protocols transition from research prototypes to realworld applications, the usability of MPC-enabled applications is increasingly critical to their successful deployment and widespread adoption. Our Web-MPC platform, designed with a focus on usability, has been deployed for privacy-preserving data aggregation initiatives with the City of Boston and the Greater Boston Chamber of Commerce. After building and deploying an initial version of the platform, we conducted a heuristic evaluation to identify usability improvements and implemented corresponding application enhancements. However, it is difficult to gauge the effectiveness of these changes within the context of real-world deployments using traditional web analytics tools without compromising the security guarantees of the platform. This work consists of two contributions that address this challenge: (1) the Web-MPC platform has been extended with the capability to collect web analytics using existing MPC protocols, and (2) as a test of this feature and a way to inform future work, this capability has been leveraged to conduct a usability study comparing the two versions ofWeb-MPC. While many efforts have focused on ways to enhance the usability of privacy-preserving technologies, this study serves as a model for using a privacy-preserving data-driven approach to evaluate and enhance the usability of privacy-preserving websites and applications deployed in realworld scenarios. Data collected in this study yields insights into the relationship between usability and security; these can help inform future implementations of MPC solutions.Published versio