A Lifecycle for User Experience Management in Agile Development

Context. Agile methods are increasingly being used by companies, to develop digital products and services faster and more effectively. Today's users not only demand products that are easy to use, but also products with a high User Experience (UX). Agile methods themselves do not directly support the development of products with a good user experience. In combination with UX activities, it is potentially possible to develop a good UX. Objective. The objective of this PhD thesis is to develop a UX Lifecycle, to manage the user experience in the context of Agile methods. With this UX Lifecycle, Agile teams can manage the UX of their product, in a targeted way. Method. We developed the UX Lifecycle step by step, according to the Design Science Research Methodology. First, we conducted a Structured Literature Review (SLR) to determine the state of the art of UX management. The result of the SLR concludes in a GAP analysis. On this basis, we derived requirements for UX management. These requirements were then implemented in the UX Lifecycle. In developing the UX Lifecycle, we developed additional methods (UX Poker, UEQ KPI, and IPA), to be used when deploying the UX Lifecycle. Each of these methods has been validated in studies, with a total of 497 respondents from three countries (Germany, England, and Spain). Finally, we validated the UX Lifecycle, as a whole, with a Delphi study, with a total of 24 international experts from four countries (Germany, Argentina, Spain, and Poland). Results. The iterative UX Lifecycle (Figure 1) consists of five steps: Initial Step 0 ‘Preparation’, Step 1 ‘UX Poker’ (before development/Estimated UX), Step 2 ‘Evaluate Prototype’ (during development/Probable UX), Step 3 ‘Evaluate Product Increment’ (after development/Implemented UX), and a subsequent Step 4 ‘UX Retrospective’. With its five steps, the UX Lifecycle provides the structure for continuously measuring and evaluating the UX, in the various phases. This makes it possible to develop the UX in a targeted manner, and to check it permanently. In addition, we have developed the UX Poker method. With this method, the User Experience can be determined by the Agile team, in the early phases of development. The evaluation study of UX Poker has indicated that UX Poker can be used to estimate the UX for user stories. In addition, UX Poker inspires a discussion about UX, that results in a common understanding of the UX of the product. To interpret the results from the evaluation of a prototype and product increment, we developed or derived the User Experience Questionnaire KPI and Importance-Performance Analysis. In a first study, we were able to successfully apply the two methods and, in combination with established UEQ methods, derive recommendations for action, regarding the improvement of the UX. This would not have been possible without their use. The results of the Delphi study, to validate the UX Lifecycle, reached consensus after two rounds. The results of the evaluation and the comments lead to the conclusion, that the UX Lifecycle has a sufficiently positive effect on UX management. Conclusion. The goal-oriented focus on UX factors and their improvement, as propagated in the UX Lifecycle, are a good way of implementing UX management in a goal-oriented manner. By comparing the results from UX Poker, the evaluation of the prototype, and product increment, the Agile team can learn more about developing a better UX, within a UX retrospective. The UX Lifecycle will have a positive effect on UX management. The use of individual components of the UX Lifecycle, such as UX Poker or Importance-Performance Analysis, already helps an Agile team to improve the user experience. But only in combination with the UX Lifecycle and the individual methods and approaches presented in this PhD thesis, is a management of the user experience in a targeted manner possible, in our view. This was the initial idea of this PhD thesis, which we are convinced we could implement

Development of a Shared UX Vision Based on UX Factors Ascertained Through Attribution

User experience (UX) is an important quality in differentiating products. For a product team, it is a challenge to develop a good positive user experience. A common UX vision for the product team supports the team in making goal-oriented decisions regarding the user experience. This paper presents an approach to developing a shared UX vision. This UX vision is developed by the product team while a collaborative session. To validate our approach, we conducted a first validation study. In this study, we conducted a collaborative session with two groups and a total of 37 participants. The group of participants comprised product managers, UX designers and comparable professional profiles. At the end of the collaborative session, participants had to fill out a questionnaire. Through questions and observations, we identified ten good practices and four bad practices in the application of our approach to developing a UX vision. The top 3 good practices mentioned by the participants include the definition of decision-making procedures (G1), determining the UX vision with the team (G2), and using general factors of the UX as a basis (G3). The top 3 bad practices are: providing too little time for the development of the UX vision (B1), not providing clear cluster designations (B2) and working without user data (B3). The results show that the present approach for developing a UX vision helps to promote a shared understanding of the intended UX in a quickly and simply way

Construction of UEQ+ scales for voice quality: measuring user experience quality of voice interaction

The UEQ+ is a modular framework for the construction of UX questionnaires. The researcher can pick those scales that fit his or her research question from a list of 16 available UX scales. Currently, no UEQ+ scales are available to allow measuring the quality of voice interactions. Given that this type of interaction is increasingly essential for the usage of digital products, this is a severe limitation of the possible products and usage scenarios that can be evaluated using the UEQ+. We describe in this paper the construction of three specific scales to measure the UX of voice interactions. Besides, we discuss how these new scales can be combined with existing UEQ+ scales in evaluation projects

Construction of a Benchmark for the User Experience Questionnaire (UEQ)

Questionnaires are a cheap and highly efficient tool for achieving a quantitative measure of a product’s user experience (UX). However, it is not always easy to decide, if a questionnaire result can really show whether a product satisfies this quality aspect. So a benchmark is useful. It allows comparing the results of one product to a large set of other products. In this paper we describe a benchmark for the User Experience Questionnaire (UEQ), a widely used evaluation tool for interactive products. We also describe how the benchmark can be applied to the quality assurance process for concrete projects

Design and Evaluation of a Short Version of the User Experience Questionnaire (UEQ-S)

The user experience questionnaire (UEQ) is a widely used questionnaire to measure the subjective impression of users towards the user experience of products. The UEQ is a semantic differential with 26 items. Filling out the UEQ takes approximately 3-5 minutes, i.e. the UEQ is already reasonably efficient concerning the time required to answer all items. However, there exist several valid application scenarios, where filling out the entire UEQ appears impractical. This paper deals with the creation of an 8 item short version of the UEQ, which is optimized for these specific application scenarios. First validations of this short version are also described

Applicability of User Experience and Usability Questionnaires

To be successful, interactive products need to fulfil user expectations and create a positive user experience (UX). An established method to measure UX involves questionnaires. What we aim in this paper is to present a list of user experience and usability questionnaires and its applicability for different digital products. A total of 13 questionnaires on usability and UX were analysed for this paper, and 25 factors were extracted from those questionnaires. A study was conducted based on this collection of factors with N=61 students. The study investigated the perceived importance of usability and UX factors for seven digital products. The goal was to have a collection of usability and UX factors that could be combined for suitable products evaluation. The results of the study revealed that no questionnaire covered all the factors perceived important by the participants

A Benchmark for the UEQ+ Framework: Construction of a Simple Tool to Quickly Interpret UEQ+ KPIs

Questionnaires are a highly efficient method to compare the user experience (UX) of different interactive products or versions of a single product. Concretely, they allow us to evaluate the UX easily and to compare different products with a numeric UX score. However, often only one UX score from a single evaluated product is available. Without a comparison to other measurements, it is difficult to interpret an individual score, e.g. to decide whether a product’s UX is good enough to compete in the market. Many questionnaires offer benchmarks to support researchers in these cases. A benchmark is the result of a larger set of product evaluations performed with the same questionnaire. The score obtained from a single product evaluation can be compared to the scores from this benchmark data set to quickly interpret the results. In this paper, the first benchmark for the UEQ+ (User Experience Questionnaire +) is presented, which was created using 3.290 UEQ+ responses for 26 successful software products. The UEQ+ is a modular framework that contains a high number of validated user experience scales that can be combined to form a UX questionnaire. Currently, no benchmark is available for this framework, making the benchmark constructed in this paper a valuable interpretation tool for UEQ+ questionnaires

Requirements for User Experience Management - A Tertiary Study

Today’s users expect to be able to interact with the products they own without much effort and also want to be excited about them. The development of a positive user experience must therefore be managed. We understand management in general as a combination of a goal, a strategy, and resources. When applied to UX, user experience management consists of a UX goal, a UX strategy, and UX resources. We conducted a tertiary study and examined the current state of existing literature regarding possible requirements. We want to figure out, what requirements can be derived from the literature reviews with the focus on UX and agile development. In total, we were able to identify and analyse 16 studies. After analysing the studies in detail, we identified different requirements for UX management. In summary, we identified 13 requirements. The most frequently mentioned requirements were prototypes and UX/usability evaluation. Communication between UX professionals and developers was identified as a major improvement in the software development process. In summary, we were able to identify requirements for UX management of People/Social, Technology/Artifacts, and Process/Practice. However, we could not identify requirements for UX management that enabled the development and achievement of a UX goal

UX Poker: Estimating the Influence of User Stories on User Experience in Early Stage of Agile Development

Agile methods are used more and more frequently to develop products by reducing development time. Requirements are typically written in user stories or epics. In this paper, a new method called UX Poker is presented. This is a method to estimate the impact of a user story on user experience before development. Thus, there is the opportunity that the product backlog can also be sorted according to the expected UX. To evaluate UX Poker, a case study was conducted with four agile teams. Besides, a workshop followed by a questionnaire was conducted with all four agile teams. The goal of being able to estimate the UX even before development was achieved. Using UX Poker to create another way to sort the product backlog can be considered achieved in this first evaluation. The results show that UX Poker can be implemented in a real- life application. Additionally, during the use of UX Poker, it was found that a shared understanding of UX began. The participants clarified in the team discussion about UX Poker what related to influence the user stories had on UX and what UX meant for their product