Improving the identification of hedonic quality in user requirements - a controlled experiment

Context and Motivation: Systematically engineering a good user experience (UX) into a computer-based system under development demands that the user requirements of the system reflect all needs, including emotional, of all stakeholders. User requirements address two different types of qualities: Pragmatic qualities (PQs), that address system functionality and usability, and hedonic qualities (HQs) that address the stakeholder's psychological well-being. Studies show that users tend to describe such satisfying UXes mainly with PQs, and that some users seem to believe that they are describing a HQ when they are actually describing a PQ. Question/Problem: The problem is to see if classification of any user requirement as PQ-related or HQ-related is difficult, and if so, why. Principal Ideas/Results: We conducted a controlled experiment in which twelve requirements-engineering and UX professionals, hereinafter called 'classifiers' classified each of 105 user requirements as PQ- related or HQ-related. The experiment shows that neither (1) a classifier's involvement in the project from which the requirements came nor (2) the classifier's use of a detailed model of the qualities in addition to the standard definitions of 'PQ' and 'HQ' has a positive effect on the consistency of the classifier's classification with that of others. Contribution: The experiment revealed that classification of user requirements is a lot harder than initially assumed