Assessing the Reliability, Validity and Adaptability of PSSUQ

Psychometrically qualified usability evaluation instruments offer many advantages to the usability practitioner. Advantages include objectivity, replicability, quantification, economy, communication, and scientific generalization. It is important that instruments used in usability evaluation have shown acceptable estimates of reliability, validity, adaptability and practicality. This paper compares the psychometric properties of the Post-Study System Usability Questionnaire (PSSUQ) instrument in Lewis’ study to a recent study that tailored the PSSUQ instrument to measure the user satisfaction of the usability of a webbased health provider interface. Lewis reported that the PSSUQ had acceptable psychometric properties. However, Lewis’ stated PSSUQ had limited generalizability and needed further examination. This research validated the PSSUQ instrument using a larger sample size in a different domain. The factor analysis and the clustering of the sub-scale items were different than Lewis’ results. Nevertheless, this research concluded that the PSSUQ instrument is adaptable and produced solid psychometric results

Qualitative assessment of Tongue Drive System by people with high-level spinal cord injury

The Tongue Drive System (TDS) is a minimally invasive, wireless, and wearable assistive technology (AT) that enables people with severe disabilities to control their environments using tongue motion. TDS translates specific tongue gestures into commands by sensing the magnetic field created by a small magnetic tracer applied to the user’s tongue. We have previously quantitatively evaluated the TDS for accessing computers and powered wheelchairs, demonstrating its usability. In this study, we focused on its qualitative evaluation by people with high-level spinal cord injury who each received a magnetic tongue piercing and used the TDS for 6 wk. We used two questionnaires, an after-scenario and a poststudy, designed to evaluate the tongue-piercing experience and the TDS usability compared with that of the sip-and-puff and the users’ current ATs. After study completion, 73% of the participants were positive about keeping the magnetic tongue-barbell in order to use the TDS. All were satisfied with the TDS performance and most said that they were able to do more things using TDS than their current ATs (4.22/5)

Effects of Perceived Prototype Fidelity in Usability Testing under Different Conditions of Observer Presence

The aim of the study was to investigate the influence of perceived prototype fidelity in usability tests by comparing two prototypes that differed with respect to their perceived proximity to the final system. The impact of the perceived developmental stage of the product was examined for participants' performance, perceived usability, emotions and psychophysiology. Eighty participants were tested, operating an electronic city guide on a mobile phone. In a 2×2×2 mixed design, the system was either presented as an early prototype or as the final system. In addition, observer presence (no observers vs. three observers) and task difficulty (high vs. low) were experimentally manipulated. Overall, the findings did not indicate major differences for perceived prototype fidelity. However, an interaction between the observer presence and prototype fidelity indicated that the observer presence had a more negative impact on the performance when testing a final system than an early prototype. Furthermore, the observer presence resulted in a psychophysiological stress response. The findings suggest that test outcomes are quite robust against different prototype perceptions but that the observer presence needs careful consideratio

The fidelity of prototype and testing environment in usability tests

This doctoral thesis investigated what setup of a usability test can best support valid test outcomes. Several aspects of contextual fidelity were manipulated in experimental usability studies, to examine their impact on test results. The first study demonstrated that the medium of prototype presentation has effects on test outcomes, which have not been found in previous research. Using a more hypothesis-driven approach, it was shown that participants exhibited more reading activity when using a paper-based as compared to a computer prototype presented on screen. This resulted in better performance, if task success required reading a short paragraph of text. Consequently, the medium of prototype presentation needs to be considered to avoid that respective usability problems go undetected. A second study demonstrated that additional observers may cause stress for test participants, which can be measured at the physiological level. Some performance indicators were affected, but only in interaction with perceived developmental stage of the test system. A third study investigated the effects of a work or leisure context on the outcome of a usability test. No effects were found for the type of usage context, but even short response time delays proved to be relevant for performance and emotions. Relevant factors for the validity of usability test outcomes were identified and theoretical and practical implications are discussed. Keywords: Usability test; paper prototype; fidelity; observer presence; work and leisure domain; system response time; heart rate variability; validity

User-centered development and testing of a monitoring system that provides feedback regarding physical functioning to elderly people

Purpose: To involve elderly people during the development of a mobile interface of a monitoring system that provides feedback to them regarding changes in physical functioning and to test the system in a pilot study. Methods and participants: The iterative user-centered development process consisted of the following phases: (1) selection of user representatives; (2) analysis of users and their context; (3) identification of user requirements; (4) development of the interface; and (5) evaluation of the interface in the lab. Subsequently, the monitoring and feedback system was tested in a pilot study by five patients who were recruited via a geriatric outpatient clinic. Participants used a bathroom scale to monitor weight and balance, and a mobile phone to monitor physical activity on a daily basis for six weeks. Personalized feedback was provided via the interface of the mobile phone. Usability was evaluated on a scale from 1 to 7 using a modified version of the Post-Study System Usability Questionnaire (PSSUQ); higher scores indicated better usability. Interviews were conducted to gain insight into the experiences of the participants with the system. Results: The developed interface uses colors, emoticons, and written and/or spoken text messages to provide daily feedback regarding (changes in) weight, balance, and physical activity. The participants rated the usability of the monitoring and feedback system with a mean score of 5.2 (standard deviation 0.90) on the modified PSSUQ. The interviews revealed that most participants liked using the system and appreciated that it signaled changes in their physical functioning. However, usability was negatively influenced by a few technical errors. Conclusion: Involvement of elderly users during the development process resulted in an interface with good usability. However, the technical functioning of the monitoring system needs to be optimized before it can be used to support elderly people in their self-management

The usability of ventilators: a comparative evaluation of use safety and user experience

Setup of the testing facilities at the Clinical Skills and Patient Simulation Center at the University of North Carolina School of Medicine, where the simulator room and observation room can be seen. (TIF 9461 kb

Psychometric Properties of the Persian Language Version of the System Usability Scale

Background: The System Usability Scale (SUS) is a widely used self–administered instrument for evaluating the usability of a wide range of products and services. The aims of this descriptive- methodological study were to develop and investigate the psychometric properties of the Iranian version of the SUS. Methods: The study was conducted among 202 university students from the Tabriz University of Medical Sciences, Iran. Content validity was evaluated by a panel of 10 experts. Construct validity was assessed by exploratory and confirmatory factor analyses. The internal consistency and test-retest reliability were assessed by Cronbach’s alpha and Intraclass Correlation Coefficient (ICC), respectively. Additionally the feasibility of the measure was judged by ceiling and floor effect. Results: Content validity of the short form of Iranian SUS was established. Factor analyses supported the conceptual uni-dimensionality, and thus confirmed the construct validity of the measure. The internal consistency (α =0.79) and test retest reliability (ICC = 0.96) were both approved and there was also no ceiling nor floor effect. Conclusions: The findings support the use of SUS for both practical and research settings in Iranian population

Iterative Design and Usability Testing of the iMHere System for Managing Chronic Conditions and Disability

A novel mobile health platform, Interactive Mobile Health and Rehabilitation (iMHere), is being developed to support wellness and self-management among people with chronic disabilities. The iMHere system currently includes a smartphone app with six modules for use by persons with disabilities and a web portal for use by medical and rehabilitation professionals or other support personnel. Our initial clinical research applying use of this system provides insight into the feasibility of employing iMHere in the development of self-management skills in young adults (ages 18-40 years) with spina bifida (Dicianno, Fairman, McCue, Parmanto, Yih, et al., 2015). This article is focused on describing the iterative design of the iMHere system including usability testing of both the app modules and clinician portal. Our pilot population of persons with spina bifida fostered the creation of a system appropriate for people with a wide variety of functional abilities and needs. As a result, the system is appropriate for use by persons with various disabilities and chronic conditions, not only spina bifida. In addition, the diversity of professionals and support personnel involved in the care of persons with spina bifida (SB) also enabled the design and implementation of the iMHere system to meet the needs of an interdisciplinary team of providers who treat various conditions. The iMHere system has the potential to foster communication and collaboration among members of an interdisciplinary healthcare team, including individuals with chronic conditions and disabilities, for client-centered approach to support self-management skills.