Users' performance in lab and non-lab environments through online usability testing:A case of evaluating the usability of digital academic libraries' websites

Abstract

The factors related to the environment in which users operate may be of a vital importance when trying to understand how they experience a particular system. It is required that we find out how we can get to know those factors to investigate if they affect the users' performance in usability testing. An online usability study has emerged that can be attempted by a large, varied pool of users' anywhere with an Internet connection. Would the usage of an online usability study help to give comprehensive insight and an understanding of the whole user experience? That is especially interesting if the user operates remotely, as we are unaware of what the users might experience while performing the test (e.g., distractions and type of device used to attempt the test). Accordingly, a pilot study of ongoing research was conducted. An identical online usability-testing tool (Loop 11) through which to apply the online usability study was used in two environments: unrestricted (the user's natural environment), and totally restricted (a simulated lab environment). Ten subjects completed the test in the restricted environment and 20 completed it in the unrestricted environment. All of the subjects were asked to perform predefined search tasks on digital libraries' websites. Their performance was analyzed and compared against the two different environments. The results showed that online usability testing is a feasible method to gain comprehensive insight into how users attempt usability testing in a non-lab environment. The results of whether different environments affect test performance show no valuable differences in most of the study's measurements. The test subjects were frequently multitask while they performed the usability testing in an unrestricted environment, but they were highly distracted if they personally interrupted. The results encourage the researcher to conduct a formal version of this study to further examine the learned lessons from the pilot study