Usability Evaluation in Virtual Environments: Classification and Comparison of Methods

Virtual environments (VEs) are a relatively new type of human-computer interface in which users perceive and act in a three-dimensional world. The designers of such systems cannot rely solely on design guidelines for traditional two-dimensional interfaces, so usability evaluation is crucial for VEs. We present an overview of VE usability evaluation. First, we discuss some of the issues that differentiate VE usability evaluation from evaluation of traditional user interfaces such as GUIs. We also present a review of VE evaluation methods currently in use, and discuss a simple classification space for VE usability evaluation methods. This classification space provides a structured means for comparing evaluation methods according to three key characteristics: involvement of representative users, context of evaluation, and types of results produced. To illustrate these concepts, we compare two existing evaluation approaches: testbed evaluation [Bowman, Johnson, & Hodges, 1999], and sequential evaluation [Gabbard, Hix, & Swan, 1999]. We conclude by presenting novel ways to effectively link these two approaches to VE usability evaluation

Demarcating mobile phone interface design guidelines to expedite selection

Guidelines are recommended as a tool for informing user interface design. Despite a proliferation of guidelines in the research literature, there is little evidence of their use in industry, nor their influence in academic literature. In this paper, we explore the research literature related to mobile phone design guidelines to find out why this should be so. We commenced by carrying out a scoping literature review of the mobile phone design guideline literature to gain insight into the maturity of the field. The question we wanted to explore was: “Are researchers building on each others’ guidelines, or is the research field still in the foundational stage?” We discovered a poorly structured field, with many researchers proposing new guidelines, but little incremental refinement of extant guidelines. It also became clear that the current reporting of guidelines did not explicitly communicate their multi-dimensionality or deployment context. This leaves designers without a clear way of discriminating between guidelines, and could contribute to the lack of deployment we observed. We conducted a thematic analysis of papers identified by means of a systematic literature review to identify a set of dimensions of mobile phone interface design guidelines. The final dimensions provide a mechanism for differentiating guidelines and expediting choice

Models for Learning (Mod4L) Final Report: Representing Learning Designs

The Mod4L Models of Practice project is part of the JISC-funded Design for Learning Programme. It ran from 1 May – 31 December 2006. The philosophy underlying the project was that a general split is evident in the e-learning community between development of e-learning tools, services and standards, and research into how teachers can use these most effectively, and is impeding uptake of new tools and methods by teachers. To help overcome this barrier and bridge the gap, a need is felt for practitioner-focused resources which describe a range of learning designs and offer guidance on how these may be chosen and applied, how they can support effective practice in design for learning, and how they can support the development of effective tools, standards and systems with a learning design capability (see, for example, Griffiths and Blat 2005, JISC 2006). Practice models, it was suggested, were such a resource. The aim of the project was to: develop a range of practice models that could be used by practitioners in real life contexts and have a high impact on improving teaching and learning practice. We worked with two definitions of practice models. Practice models are: 1. generic approaches to the structuring and orchestration of learning activities. They express elements of pedagogic principle and allow practitioners to make informed choices (JISC 2006) However, however effective a learning design may be, it can only be shared with others through a representation. The issue of representation of learning designs is, then, central to the concept of sharing and reuse at the heart of JISC’s Design for Learning programme. Thus practice models should be both representations of effective practice, and effective representations of practice. Hence we arrived at the project working definition of practice models as: 2. Common, but decontextualised, learning designs that are represented in a way that is usable by practitioners (teachers, managers, etc).(Mod4L working definition, Falconer & Littlejohn 2006). A learning design is defined as the outcome of the process of designing, planning and orchestrating learning activities as part of a learning session or programme (JISC 2006). Practice models have many potential uses: they describe a range of learning designs that are found to be effective, and offer guidance on their use; they support sharing, reuse and adaptation of learning designs by teachers, and also the development of tools, standards and systems for planning, editing and running the designs. The project took a practitioner-centred approach, working in close collaboration with a focus group of 12 teachers recruited across a range of disciplines and from both FE and HE. Focus group members are listed in Appendix 1. Information was gathered from the focus group through two face to face workshops, and through their contributions to discussions on the project wiki. This was supplemented by an activity at a JISC pedagogy experts meeting in October 2006, and a part workshop at ALT-C in September 2006. The project interim report of August 2006 contained the outcomes of the first workshop (Falconer and Littlejohn, 2006). The current report refines the discussion of issues of representing learning designs for sharing and reuse evidenced in the interim report and highlights problems with the concept of practice models (section 2), characterises the requirements teachers have of effective representations (section 3), evaluates a number of types of representation against these requirements (section 4), explores the more technically focused role of sequencing representations and controlled vocabularies (sections 5 & 6), documents some generic learning designs (section 8.2) and suggests ways forward for bridging the gap between teachers and developers (section 2.6). All quotations are taken from the Mod4L wiki unless otherwise stated

InfoVis experience enhancement through mediated interaction

Information visualization is an experience in which both the aesthetic representations and interaction are part. Such an experience can be augmented through close consideration of its major components. Interaction is crucial to the experience, yet it has seldom been adequately explored in the field. We claim that direct mediated interaction can augment such an experience. This paper discusses the reasons behind such a claim and proposes a mediated interactive manipulation scheme based on the notion of directness. It also describes the ways in which such a claim will be validated. The Literature Knowledge Domain (LKD) is used as the concrete domain around which the discussions will be held

Participatory Scenario Generation: Communicating Usability Issues in Product Design through User Involvement in Scenario Generation\ud

Scenarios have proven to be a valuable tool in evaluating and communicating usability issues in consumer product design. Scenarios are explicit descriptions of hypothetical use situations. Realistic scenarios can serve as a valuable frame of reference to evaluate design solutions with regard to usability. To be able to achieve this required level of realism, involving users in scenario generation is essential. In this presentation we discuss how and where users can be involved in a scenario based product design process by means of examples of design projects that were executed by master students Industrial Design Engineering of the University of Twente. \ud \ud We distinguish direct and indirect scenario generation. In direct scenario generation the user is actively involved in a participatory scenario generation session: the scenarios are created together with users. Indirect scenario generation is an approach in which scenarios are created by designers based on common analysis techniques like observations and interviews. These scenarios are then offered to users for confirmation. Both types of user involvement in scenario generation can be aimed at either current use scenarios which describe the current situation or future use scenarios which include a new product design. \ud \ud The examples show that all strategies can be applied successfully to create realistic scenarios. Which strategy to choose depends among others upon risks and privacy issues, occurrence of infrequent events and availability of users. Furthermore, the variety of approaches shows that there is still a lot to explore with regard to benefits and limitations of the many techniques that can be applied in generating scenarios for consumer product design. We hope to contribute to this field by means of the research in our group and the work of students in the SBPD course\u

Towards a Tool-based Development Methodology for Pervasive Computing Applications

Despite much progress, developing a pervasive computing application remains a challenge because of a lack of conceptual frameworks and supporting tools. This challenge involves coping with heterogeneous devices, overcoming the intricacies of distributed systems technologies, working out an architecture for the application, encoding it in a program, writing specific code to test the application, and finally deploying it. This paper presents a design language and a tool suite covering the development life-cycle of a pervasive computing application. The design language allows to define a taxonomy of area-specific building-blocks, abstracting over their heterogeneity. This language also includes a layer to define the architecture of an application, following an architectural pattern commonly used in the pervasive computing domain. Our underlying methodology assigns roles to the stakeholders, providing separation of concerns. Our tool suite includes a compiler that takes design artifacts written in our language as input and generates a programming framework that supports the subsequent development stages, namely implementation, testing, and deployment. Our methodology has been applied on a wide spectrum of areas. Based on these experiments, we assess our approach through three criteria: expressiveness, usability, and productivity

Heuristic usability evaluation on games: a modular approach

Heuristic evaluation is the preferred method to assess usability in games when experts conduct this evaluation. Many heuristics guidelines have been proposed attending to specificities of games but they only focus on specific subsets of games or platforms. In fact, to date the most used guideline to evaluate games usability is still Nielsen’s proposal, which is focused on generic software. As a result, most evaluations do not cover important aspects in games such as mobility, multiplayer interactions, enjoyability and playability, etc. To promote the usage of new heuristics adapted to different game and platform aspects we propose a modular approach based on the classification of existing game heuristics using metadata and a tool, MUSE (Meta-heUristics uSability Evaluation tool) for games, which allows a rebuild of heuristic guidelines based on metadata selection in order to obtain a customized list for every real evaluation case. The usage of these new rebuilt heuristic guidelines allows an explicit attendance to a wide range of usability aspects in games and a better detection of usability issues. We preliminarily evaluate MUSE with an analysis of two different games, using both the Nielsen’s heuristics and the customized heuristic lists generated by our tool.Unión Europea PI055-15/E0

Emergent digital services in public libraries : a domain study

Purpose: This paper explores the emergence of digital services in the public library domain via an extensive study of the websites of all Scottish public library services Design/methodology/approach: In a 4 month period all 32 of Scotland’s public library authority websites were visited by a researcher. The goal of the researcher was to record the options available from the library homepages in the following way: •Role of library in providing page content: content provider or access provider? •Was the page providing a digital service? •What was the audience for the page? Adult, child, or not specified? •Description of page content •Any noted usability issues Each site was only visited to three levels below that of the initial homepage. Findings: The study found a good standard of innovation in digital services around LMS functions, offering users the ability to keep in control of their borrowing and reserving. In addition there was a consistent set of electronic reference resources subscribed to by multiple libraries, offering high quality information both within the library and for library members from their home or workplace. Problems were found with regards to guidance on the usage of these resources, as well as confusion and inconsistency in terminology usage across different library services. Research limitations/implications: The paper examines only Scottish public library sites, thus can only claim to be representative of that country. It also can only represent the sites at the time they were examined. Practical implications: The paper should be of interest to public and other librarians interested in patterns across web sites in their sector. Originality/value: This is the first national study of Scottish public library websites and its findings should be of value as a result