Development and Validation of a Positively Worded German Version of the System Usability Scale: The SUS-DE-Pos

Despite its popularity, knowledge of the psychometric quality of the System Usability Scale (SUS) in German is limited. This paper addresses this by developing a positively worded German SUS version and investigating two existing German versions. In a preregistered first study, initial positive alternatives for the negative items were evaluated, and the two German versions were compared. Based on these results, a preregistered second study validated the new positive-only SUS version and independently validated the two original German SUS versions. Results indicated that the SUS-DE-Rummel-pos, a positive version based on Rummel (2016), performed best. Researchers are thus advised to use this version. For consistency, if negative items are needed, Rummel (2016)'s version is recommended. Overall, this work provides validated German SUS versions, both with and without negative items, addressing a critical gap for German-speaking HCI researchers and practitioners

Independent Validation of the Video Game Dispositional Flow Scale With League of Legends Players

Flow is a highly influential concept across many research domains. Because of the close links between enjoyment and flow, it is also a central concept in game research. Accordingly, many measurement instruments in the form of questionnaires have been developed to measure flow in video games. However, limited independent validation work has been carried out to date. In this project, we evaluated the recently developed Video Game Dispositional Flow Scale (VGDFS) in the context of League of Legends. An online survey posted on the online community reddit.com yielded a sample of N = 140. Confirmatory factor analysis did not support the originally proposed structure. Subsequent exploratory factor analysis resulted in inconclusive findings. Convergent validity with other questionnaires for the measurement of interest, enjoyment, and flow could be demonstrated in our data, although with unusually high correlations between the VGDFS and another flow measure. For divergent validity, correlations with questionnaires for the measurement of boredom and trait anxiety in the hypothesized direction could be found. Findings suggest that the VGDFS needs further evaluation in different video game contexts while also hinting at measurement problems possibly related to conceptual issues with flow

Measurement practices in user experience (UX) research: a systematic quantitative literature review

User experience (UX) research relies heavily on survey scales to measure users' subjective experiences with technology. However, repeatedly raised concerns regarding the improper use of survey scales in UX research and adjacent fields call for a systematic review of current measurement practices. Therefore, we conducted a systematic literature review, screening 153 papers from four years of the ACM Conference on Human Factors in Computing Systems proceedings (ACM CHI 2019 to 2022), of which 60 were eligible empirical studies using survey scales to study users' experiences. We identified 85 different scales and 172 distinct constructs measured. Most scales were used once (70.59%), and most constructs were measured only once (66.28%). The System Usability Scale was the most popular scale, followed by the User Experience Questionnaire, and the NASA Task Load Index. Regarding constructs, usability was the most frequently measured, followed by attractiveness, effort, and presence. Furthermore, results show that papers rarely contained complete rationales for scale selection (20.00%) and seldom provided all scale items used (30.00%). More than a third of all scales were adapted (34.19%), while only one-third of papers reported any scale quality investigation (36.67%). On the basis of our results, we highlight questionable measurement practices in UX research and suggest opportunities to improve scale use for UX-related constructs. Additionally, we provide six recommended steps to promote enhanced rigor in following best practices for scale-based UX research