Effects of Perceived Prototype Fidelity in Usability Testing under Different Conditions of Observer Presence

The aim of the study was to investigate the influence of perceived prototype fidelity in usability tests by comparing two prototypes that differed with respect to their perceived proximity to the final system. The impact of the perceived developmental stage of the product was examined for participants' performance, perceived usability, emotions and psychophysiology. Eighty participants were tested, operating an electronic city guide on a mobile phone. In a 2×2×2 mixed design, the system was either presented as an early prototype or as the final system. In addition, observer presence (no observers vs. three observers) and task difficulty (high vs. low) were experimentally manipulated. Overall, the findings did not indicate major differences for perceived prototype fidelity. However, an interaction between the observer presence and prototype fidelity indicated that the observer presence had a more negative impact on the performance when testing a final system than an early prototype. Furthermore, the observer presence resulted in a psychophysiological stress response. The findings suggest that test outcomes are quite robust against different prototype perceptions but that the observer presence needs careful consideratio

The fidelity of prototype and testing environment in usability tests

This doctoral thesis investigated what setup of a usability test can best support valid test outcomes. Several aspects of contextual fidelity were manipulated in experimental usability studies, to examine their impact on test results. The first study demonstrated that the medium of prototype presentation has effects on test outcomes, which have not been found in previous research. Using a more hypothesis-driven approach, it was shown that participants exhibited more reading activity when using a paper-based as compared to a computer prototype presented on screen. This resulted in better performance, if task success required reading a short paragraph of text. Consequently, the medium of prototype presentation needs to be considered to avoid that respective usability problems go undetected. A second study demonstrated that additional observers may cause stress for test participants, which can be measured at the physiological level. Some performance indicators were affected, but only in interaction with perceived developmental stage of the test system. A third study investigated the effects of a work or leisure context on the outcome of a usability test. No effects were found for the type of usage context, but even short response time delays proved to be relevant for performance and emotions. Relevant factors for the validity of usability test outcomes were identified and theoretical and practical implications are discussed. Keywords: Usability test; paper prototype; fidelity; observer presence; work and leisure domain; system response time; heart rate variability; validity