The fidelity of prototype and testing environment in usability tests

This doctoral thesis investigated what setup of a usability test can best support valid test outcomes. Several aspects of contextual fidelity were manipulated in experimental usability studies, to examine their impact on test results. The first study demonstrated that the medium of prototype presentation has effects on test outcomes, which have not been found in previous research. Using a more hypothesis-driven approach, it was shown that participants exhibited more reading activity when using a paper-based as compared to a computer prototype presented on screen. This resulted in better performance, if task success required reading a short paragraph of text. Consequently, the medium of prototype presentation needs to be considered to avoid that respective usability problems go undetected. A second study demonstrated that additional observers may cause stress for test participants, which can be measured at the physiological level. Some performance indicators were affected, but only in interaction with perceived developmental stage of the test system. A third study investigated the effects of a work or leisure context on the outcome of a usability test. No effects were found for the type of usage context, but even short response time delays proved to be relevant for performance and emotions. Relevant factors for the validity of usability test outcomes were identified and theoretical and practical implications are discussed. Keywords: Usability test; paper prototype; fidelity; observer presence; work and leisure domain; system response time; heart rate variability; validity

The Influence of Product Aesthetics and Usability over the Course of Time: A Longitudinal Field Experiment

A longitudinal field experiment was carried out over a period of two weeks to examine the influence of product aesthetics and inherent product usability. A 2 x 2 x 3 mixed design was used in the study, with product aesthetics (high / low) and usability (high / low) being manipulated as between-subjects variables and exposure time as a repeated-measures variable (3 levels). A sample of 60 mobile phone users was tested during a multiple-session usability test. A range of outcome variables was measured, including performance, perceived usability, perceived aesthetics, and emotion. A major finding was that the positive effect of an aesthetically appealing product on perceived usability, reported in many previous studies, began to wane with increasing exposure time. The data provided similar evidence for emotion, which also showed changes as a function of exposure time. The study has methodological implications for the future design of usability tests, notably suggesting the need for longitudinal approaches in usability research