Evaluating the usability and usefulness of a digital library

Purpose - System usability and system usefulness are interdependent properties of system interaction, which in combination, determine system satisfaction and usage. Often approached separately, or in the case of digital libraries, often focused upon usability, there is emerging consensus among the research community for their unified treatment and research attention. However, a key challenge is to identify, both respectively and relatively, what to measure and how, compounded by concerns regarding common understanding of usability measures, and associated calls for more valid and complete measures within integrated and comprehensive models. The purpose of this paper is to address this challenge. Design/methodology/approach - Identified key usability and usefulness attributes and associated measures, compiled an integrated measurement framework, identified a suitable methodological approach for application of the framework, and conducted a pilot study on an interactive search system developed by a Health Service as part of their e-library service. Findings - Effectiveness, efficiency, aesthetic appearance, terminology, navigation, and learnability are key attributes of system usability; and relevance, reliability, and currency key attributes of system usefulness. There are shared aspects to several of these attributes, but each is also sufficiently unique to preserve its respective validity. They can be combined as part of a multi-method approach to system evaluation. Research limitations/implications - Pilot study has demonstrated that usability and usefulness can be readily combined, and that questionnaire and observation are valid multi-method approaches, but further research is called for under a variety of conditions, with further combinations of methods, and larger samples. Originality/value - This paper provides an integrated measurement framework, derived from the goal, question, metric paradigm, which provides a relatively comprehensive and representative set of system usability and system usefulness attributes and associated measures, which could be adapted and further refined on a case-by-case basis

Analytical Evaluation of Groupware Usability in Concerted Work Scenarios

This paper addresses the usability evaluation of groupware tools supporting teams working in an intensive concerted effort towards a shared goal. Considering the evaluation of this technology in such scenario is a complex and costly endeavor, we propose a discount analytical evaluation approach. Our approach uses available models of human performance to estimate groupware usability. The paper illustrates the use of the Keystroke-Level Model for analyzing a concerted work scenario and shows how the adopted evaluation method affords analytical experiments with alternative groupware designs. This paper contributes to understand the fine-grained details involved in groupware desig

How do design and evaluation interrelate in HCI research?

Presented at DIS 2006, the Symposium on Designing Interactive Systems, the 6th ACM conference on Designing Interactive Systems, University Park, PA, DOI: http://dx.doi.org/10.1145/1142405.1142421Human-Computer Interaction (HCI) is defined by the Association for Computing Machinery (ACM) Special Interest Group on Computer-Human Interaction (SIGCHI) as “a discipline concerned with the design, evaluation, and implementation of interactive computing systems for human use and with the study of the major phenomenon surrounding them” [18]. In HCI there are authors that focus more on designing for usability and there are authors that focus more on evaluating usability. The relationship between these communities is not really clear. We use author cocitation analysis, multivariate techniques, and visualization tools to explore the relationships between these communities. The results of the analysis revealed seven clusters that could be identified as Design Theory and Complexity, Design Rationale, Cognitive Theories and Models, Cognitive Engineering, Computer-Supported Cooperative Work (CSCW), Participatory Design, and User-Centered Design

Quantitative Analysis of Shared Workspace Usability

In this paper we propose a discount analytical approach for quantitatively evaluating shared workspace usability. We started with existing models of human performance, developed in the Human-Computer Interaction field and thus focused on single user interactions, and studied the benefits of extending them to collaborative scenarios. The obtained results indicate that the proposed approach: (1) facilitates the fine-grained analysis of intensive concerted work scenarios; (2) provides quantitative estimates of collaborative actions performed in shared workspaces; and (3) affords comparing alternative design decisions, using shared workspace usability metrics derived from the aforementioned quantitative estimate

Maintainer perspectives on data-driven transport asset management and the future role of the Internet-of-Things

In this paper we present fieldwork findings from engagements with a cooperative group of strategic and operations transport maintainers, responsible for the maintenance of drainage infrastructure to mitigate the risk and impact of flooding on the network. Through transport maintainer perspectives from rail and highways sectors, we focus on developing an understanding of work-practices in the context of transport surface-water management, and how the recent acceleration towards ‘data-driven’ technologies in support of maintenance intervention decision-making (i.e. the work of coordinating the cleaning of drainage) is integrated and impacting on current work-practice. Furthermore, we document and consider how maintainers perceive the potential role of the Internet-of-Things (IoT) and highlight emerging opportunities and tensions that may arise ahead of its future design and implementation

Mixed reality architecture

This thesis develops and investigates Mixed Reality Architectures (MRA), dynamic shared architectural topologies, which span physical and virtual spaces. A theoretical framework is developed to describe the field of possible architectures. As the result of a first pilot study, this is then extended with the concept of the Mixed Reality Architectural Cell (MRACell). MRACells consist of one physical and one virtual space, linked by a two-way video and audio connection. The video of a real physical space is rendered on an MRACell, which can move within the virtual environment. A projector and screen in the real space renders an image of the virtual environment from the point of view of that MRACell. Inhabitants can move their MRACell in relation to all others within the shared virtual environment, allowing ad hoc as well as planned remote social interaction. In this sense MRACells can be described as novel architectural interfaces extending real physical space, via a shared virtual environment to link to other real spaces. An in-depth study lasting one year and involving six office-based MRACells, used video recordings, the analysis of event logs, diaries and an interview survey. This produced a series of ethnographic vignettes describing social interaction within MRA in detail. The study found that the MRA was effective at supporting remote social interaction between users. Usage patterns appeared to be motivated by awareness and communication or conversely privacy requirements. This usage maintained and strengthened social ties. Social interaction was both visible to others and part of the everyday activities at the respective office spaces. It resulted from the virtual adjacencies introduced by MRA that allowed the ‘spatial’ integration of remote locations. However, the virtual spatial framework making this possible, introduced new topological limitations on the number of concurrent connections that were available. Overall, it was found that the dynamic architectural topology directly affected social interaction, while social interaction itself re-shaped the topology. These findings are of direct relevance to current developments, which aim to use communications media to overcome the spatial dispersion of work groups in modern organizations. Finally, the differences in use that were observed between groups of inhabitants suggest that spatial cognition in Mixed Reality is affected both by the interface technology and by the social practices surrounding it. In response, it is suggested that in order to investigate the new generation of mixed physical and virtual technologies, cognitive science should take into account their affordances as ‘virtual extensions’ to both our bodies and to our environment

Context-centered design: bridging the gap between designing and understanding

The design of Electronic Medical Record (EMR) systems and other clinical information systems challenges traditional HCI design in the following two ways: (a) system interactions involve multiple clinician/non-clinician teams, various interactive medical devices and medical artifacts in highly mobilized contexts, and (b) it is extremely difficult for designers to understand the highly knowledge-intensive clinical medicine field and design a system to fit into the hospital environment. Past literature suggests that context of system use could potentially solve these EMR design challenges. To enhance system design quality and bridge designers’ expertise and end-users’ domain understanding, we developed an operational method called Context-Centered Framework and carried out an empirical study to test the effectiveness of it. The empirical study examined the impact of the framework on a mobilized nursing task using scenario-based design and claims analysis approaches. The results indicated that designers improved their understanding towards the clinicians’ working environment and incorporated more usability concerns in their design product through the use of the Context-Centered Framework. This suggests that focusing on the context of system use could improve the quality of design for the systems situated in the highly complex, mobile and ubiquitous environment and could benefit clinicians’ practice.Ph.D., Information Studies -- Drexel University, 200