Evaluation of the Perceived Persuasiveness Questionnaire:User-Centered Card-Sort Study

Abstract

Background: eHealth technologies aim to change users’ health-related behavior. Persuasive design and system features can make an eHealth technology more motivating, engaging, or supportive to its users. The Persuasive Systems Design (PSD) model incorporates software features that have the possibility to increase the persuasiveness of technologies. However, the effects of specific PSD software features on the effectiveness of an intervention are still largely unknown. The Perceived Persuasiveness Questionnaire (PPQ) was developed to gain insight into the working mechanisms of persuasive technologies. Although the PPQ seems to be a suitable method for measuring subjective persuasiveness, it needs to be further evaluated to determine how suitable it is for measuring perceived persuasiveness among the public. Objective: This study aims to evaluate the face and construct validity of the PPQ, identify points of improvement, and provide suggestions for further development of the PPQ. Methods: A web-based closed-ended card-sort study was performed wherein participants grouped existing PPQ items under existing PPQ constructs. Participants were invited via a Massive Open Online Course on eHealth. A total of 398 people (average age 44.15 years, SD 15.17; 251/398, 63.1% women) completed the card sort. Face validity was evaluated by determining the item-level agreement of the original PPQ constructs. Construct validity was evaluated by determining the construct in which each item was placed most often, regardless of the original placement and how often 2 items were (regardless of the constructs) paired together and what interitem correlations were according to a cluster analysis. Results: Four PPQ constructs obtained relatively high face validity scores: perceived social support, use continuance, perceived credibility, and perceived effort. Item-level agreement on the other constructs was relatively low. Item-level agreement for almost all constructs, except perceived effort and perceived effectiveness, would increase if items would be grouped differently. Finally, a cluster analysis of the PPQ indicated that the strengths of the newly identified 9 clusters varied strongly. Unchanged strong clusters were only found for perceived credibility support, perceived social support, and use continuance. The placement of the other items was much more spread out over the other constructs, suggesting an overlap between them. Conclusions: The findings of this study provide a solid starting point toward a redesigned PPQ that is a true asset to the field of persuasiveness research. To achieve this, we advocate that the redesigned PPQ should adhere more closely to what persuasiveness is according to the PSD model and to the mental models of potential end users of technology. The revised PPQ should, for example, enquire if the user thinks anything is done to provide task support but not how this is done exactly