Usability problem reports for comparative studies: consistency and inspectability

The present study explores issues of consistency and inspectability in usability test data analysis processes and reports. Problem reports resulting from usability tests performed by three professional usability labs in three different countries are compared. Each of the labs conducted a usability test on the same product, applying an agreed test protocol that was collaboratively developed by the labs. Each lab first analyzed their own findings as they always do in their regular professional practice. A few weeks later, they again analyzed their findings but then everyone applied the same method (SlimDEVAN: a simplified version of DEVAN, a method developed for facilitating comparison of findings from usability tests in an academic setting). It was found that levels of agreement between labs did not improve when they all used SlimDEVAN there was inherent subjectivity in their analyses. It was found that consistency of single analysts in analyzing their data can be improved by using a method like SlimDEVAN. Such methods can also help in making the analysis process and findings more inspectable. Inspectability is helpful in comparative studies based on identified usability problems because it allows for the traceability of findings to original observations, as well as for laying bare the subjective parts of the data analysis

Design-inclusive UX research: design as a part of doing user experience research

Since the third wave in human–computer interaction (HCI), research on user experience (UX) has gained momentum within the HCI community. The focus has shifted from systematic usability requirements and measures towards guidance on designing for experiences. This is a big change, since design has traditionally not played a large role in HCI research. Yet, the literature addressing this shift in focus is very limited. We believe that the field of UX research can learn from a field where design and experiential aspects have always been important: design research. In this article, we discuss why design is needed in UX research and how research that includes design as a part of research aimed at supporting and advancing UX design practice. We do this by investigating types of design-inclusive UX research and by learning from real-life cases of UX-related design research. We report the results of an interview study with 41 researchers in three academic research units where design research meets UX research. Based on our interview findings, and building on existing literature, we describe the different roles design can play in research projects. We also report how design research results can inform designing for experience methodologically or by providing new knowledge on UX. The results are presented in a structured palette that can help UX researchers reflect and focus more on design in their research projects, thereby tackling experience design challenges in their own research.acceptedVersionPeer reviewe

Video-mediated Participation in Virtual Museum Tours for Older Adults

This paper introduces a virtual tour, Visit the Louvre, designed specifically to engage older adults in an immersive visit through part of the Louvre by a distant real-life guide. An initial diary study and a creative workshop were conducted to understand the needs and values of older adults and how to support participation to virtual museum visits with a video-based communication system. Preliminary results show that ‘virtual visitors’ experienced high levels of social and spatial presence; immersion and engagement were quite high independent of the level of interactivity of the guide, or the presence of others. QC 20170110</p

Assessing usability evaluation methods on their effectiveness to elicit verbal comments from children subjects

An exploratory study is described looking at children's ability to provide verbal comments in usability evaluation sessions. Six evaluation methods were applied to test an interactive toy by children aged 6 and 7. The results show that most verbal comments were gathered during Active Intervention sessions, by asking children questions during tasks. Unexpectedly, the Co-Discovery sessions were less successful, because children did not collaborate very well. Children also provided useful comments in the Thinking Aloud, Retrospection, and Peer Tutoring sessions. They could reflect on their actions at the end of Retrospection sessions, and were able to teach other children how to interact with the toy in Peer Tutoring sessions

Relevance by play: an integrated framework for designing museum experiences

The notion of relevance is often used as a concept to be considered for making a museum matter to its visitors. The term, however, is rarely operationalized for use by designers, practitioners, or scientists in their work on museum experiences. We propose an integrated framework for designing relevant museum experiences, in which we distinguish between four stages of seeding and growing relevance in new audiences, called “trigger”, “engage”, “consolidate” and “relate“. The framework proposes to see designing for relevance as developing ways of integrating meaning-making, play and acceptable visitor effort across all these stages. It is intended to provide sensitizing concepts for use in further research on designing for relevance, as well as in design-related activities such as crafting requirements for new museum experiences, analyzing existing museum experiences and developing new museum experiences

Assessing usability evaluation methods on their effectiveness to elicit verbal comments from children subjects

An exploratory study is described looking at children's ability to provide verbal comments in usability evaluation sessions. Six evaluation methods were applied to test an interactive toy by children aged 6 and 7. The results show that most verbal comments were gathered during Active Intervention sessions, by asking children questions during tasks. Unexpectedly, the Co-Discovery sessions were less successful, because children did not collaborate very well. Children also provided useful comments in the Thinking Aloud, Retrospection, and Peer Tutoring sessions. They could reflect on their actions at the end of Retrospection sessions, and were able to teach other children how to interact with the toy in Peer Tutoring sessions.</p

Assessing usability evaluation methods on their effectiveness to elicit verbal comments from children subjects

An exploratory study is described looking at children's ability to provide verbal comments in usability evaluation sessions. Six evaluation methods were applied to test an interactive toy by children aged 6 and 7. The results show that most verbal comments were gathered during Active Intervention sessions, by asking children questions during tasks. Unexpectedly, the Co-Discovery sessions were less successful, because children did not collaborate very well. Children also provided useful comments in the Thinking Aloud, Retrospection, and Peer Tutoring sessions. They could reflect on their actions at the end of Retrospection sessions, and were able to teach other children how to interact with the toy in Peer Tutoring sessions.</p