A Lifecycle for User Experience Management in Agile Development

Context. Agile methods are increasingly being used by companies, to develop digital products and services faster and more effectively. Today's users not only demand products that are easy to use, but also products with a high User Experience (UX). Agile methods themselves do not directly support the development of products with a good user experience. In combination with UX activities, it is potentially possible to develop a good UX. Objective. The objective of this PhD thesis is to develop a UX Lifecycle, to manage the user experience in the context of Agile methods. With this UX Lifecycle, Agile teams can manage the UX of their product, in a targeted way. Method. We developed the UX Lifecycle step by step, according to the Design Science Research Methodology. First, we conducted a Structured Literature Review (SLR) to determine the state of the art of UX management. The result of the SLR concludes in a GAP analysis. On this basis, we derived requirements for UX management. These requirements were then implemented in the UX Lifecycle. In developing the UX Lifecycle, we developed additional methods (UX Poker, UEQ KPI, and IPA), to be used when deploying the UX Lifecycle. Each of these methods has been validated in studies, with a total of 497 respondents from three countries (Germany, England, and Spain). Finally, we validated the UX Lifecycle, as a whole, with a Delphi study, with a total of 24 international experts from four countries (Germany, Argentina, Spain, and Poland). Results. The iterative UX Lifecycle (Figure 1) consists of five steps: Initial Step 0 ‘Preparation’, Step 1 ‘UX Poker’ (before development/Estimated UX), Step 2 ‘Evaluate Prototype’ (during development/Probable UX), Step 3 ‘Evaluate Product Increment’ (after development/Implemented UX), and a subsequent Step 4 ‘UX Retrospective’. With its five steps, the UX Lifecycle provides the structure for continuously measuring and evaluating the UX, in the various phases. This makes it possible to develop the UX in a targeted manner, and to check it permanently. In addition, we have developed the UX Poker method. With this method, the User Experience can be determined by the Agile team, in the early phases of development. The evaluation study of UX Poker has indicated that UX Poker can be used to estimate the UX for user stories. In addition, UX Poker inspires a discussion about UX, that results in a common understanding of the UX of the product. To interpret the results from the evaluation of a prototype and product increment, we developed or derived the User Experience Questionnaire KPI and Importance-Performance Analysis. In a first study, we were able to successfully apply the two methods and, in combination with established UEQ methods, derive recommendations for action, regarding the improvement of the UX. This would not have been possible without their use. The results of the Delphi study, to validate the UX Lifecycle, reached consensus after two rounds. The results of the evaluation and the comments lead to the conclusion, that the UX Lifecycle has a sufficiently positive effect on UX management. Conclusion. The goal-oriented focus on UX factors and their improvement, as propagated in the UX Lifecycle, are a good way of implementing UX management in a goal-oriented manner. By comparing the results from UX Poker, the evaluation of the prototype, and product increment, the Agile team can learn more about developing a better UX, within a UX retrospective. The UX Lifecycle will have a positive effect on UX management. The use of individual components of the UX Lifecycle, such as UX Poker or Importance-Performance Analysis, already helps an Agile team to improve the user experience. But only in combination with the UX Lifecycle and the individual methods and approaches presented in this PhD thesis, is a management of the user experience in a targeted manner possible, in our view. This was the initial idea of this PhD thesis, which we are convinced we could implement

User Experience Analysis on Student Services Website using User Experience Questionnaire (UEQ) KPI and Importance Performance Analysis (IPA) (Case Study: UPN "Veteran" Yogyakarta)

Purpose: This study aims to obtain an end-user assessment of User Experience on the Student Services website so that it can be used as a priority material for improvement for the Bureau of Academic, Student Affairs, Planning, and Cooperation (AKPK) of the National Development University (UPN) "Veteran" Yogyakarta, when developing a website further.Design/methodology/approach: The User Experience Assessment on the Student Services website refers to 6 aspects of the assessment of the User Experience Questionnaire (UEQ) KPI method. The existing results will be mapped into an IPA (Importance Performance Analysis) diagram.Findings/result: The results of user experience testing on the Student Services website using the UEQ method, get the Good category for the Efficiency (1.56) and Dependability (1.57) aspects, the Above average category for the Attractiveness aspect (1.28), Perspicuity (1.57), and Stimulation (1.15) and the Bad category on Novelty (-0.27). Mapping the results of the UEQ KPI to the IPA quadrant, getting the results of the Attractiveness, Perspicuity, Efficiency, and Dependability aspects are in Quadrant 1, the Stimulation aspect is in Quadrant 2, the Novelty aspect is in Quadrant 3, and no aspect is in Quadrant 4. Based on the results of the study, it can be concluded that the user experience value of the end user is good. Recommendations for improvement priorities for the Student Services website can further prioritize Novelty aspels that are in Quadrant 3 and in Bad condition.Originality/value/state of the art: The focus of this research is the same as previous research, namely analyzing User Experience with reference to the assessment aspects of the KPI User Experience Questionnaire (UEQ) and IPA (Importance Performance Analysis) methods. The difference that can be seen in this study is from the implementation of the method into different case studies with the objectives and urgency and problems described in accordance with the existing research object

Redesain Tampilan Website Repository Kampus Institut Teknologi Telkom Purwokerto dan Pengujian Menggunakan Usability Testing

Berdasarkan hasil dari survey awal yang telah dibuat oleh peneliti pada tanggal 18 mei 2022 dengan metode random sampling sebagai data penelitian awal, 11 orang (78,6%) sangat setuju diadakannya perubahan tampilan website repository kampus ITTP dan 3 orang (21,4%) setuju diadakannya perubahan website repository kampus ITTP. Atas dasar permasalahan tersebut, penelitian dilakukan untuk menilai pengalaman pengguna (UX) situs web untuk repositori kampus ITTP. Tujuan studi ini adalah untuk mengevaluasi kinerja sistem berdasarkan umpan balik pengguna pada perombakan situs web untuk repositori kampus ITTP. Proses evaluasi user experience menggunakan metode usability testing dengan moderated remote usability testing dan user experience questionnaire (UEQ). Hasil penelitian moderated usability testing menunjukkan (completion rate) yaitu 96% yang berarti hampir dari semua task yang diujikan berhasil dikerjakan oleh para responden dengan tingkat kesuksesan yang tinggi. Sedangkan Time-based Efficiency menunjukkan dengan nilai 4,6 goal/sec. Sedangkan hasil pengujian dengan UEQ menunjukkan bahwa website memiliki hasil evaluasi yang positif. Hasil dari benchmark UEQ menunjukkan nilai 5 kategori “Excellent” dan 1 kategori mendapatkan nilai “Good”. Hasil Skala Likert mendapatkan hasil 90% yang masuk dalam kategori “ Sangat Layak ” dalam range  (81% - 100%). Perombakan situs web repositori kampus ittp sudah berada pada tingkat pengalaman pengguna yang sangat baik, menurut peringkat pengalaman pengguna secara keseluruhan

A Benchmark for the UEQ+ Framework: Construction of a Simple Tool to Quickly Interpret UEQ+ KPIs

Questionnaires are a highly efficient method to compare the user experience (UX) of different interactive products or versions of a single product. Concretely, they allow us to evaluate the UX easily and to compare different products with a numeric UX score. However, often only one UX score from a single evaluated product is available. Without a comparison to other measurements, it is difficult to interpret an individual score, e.g. to decide whether a product’s UX is good enough to compete in the market. Many questionnaires offer benchmarks to support researchers in these cases. A benchmark is the result of a larger set of product evaluations performed with the same questionnaire. The score obtained from a single product evaluation can be compared to the scores from this benchmark data set to quickly interpret the results. In this paper, the first benchmark for the UEQ+ (User Experience Questionnaire +) is presented, which was created using 3.290 UEQ+ responses for 26 successful software products. The UEQ+ is a modular framework that contains a high number of validated user experience scales that can be combined to form a UX questionnaire. Currently, no benchmark is available for this framework, making the benchmark constructed in this paper a valuable interpretation tool for UEQ+ questionnaires

Application of Voice Personal Assistants in the Context of Smart University

Los asistentes personales de voz basados en técnicas avanzadas de comprensión del lenguaje natural se muestran como un recurso prometedor frente al reto del diseño de plataformas virtuales de aprendizaje. Específicamente, estos recursos pueden servir de apoyo para la mejora del proceso de enseñanza-aprendizaje. El objetivo principal de este trabajo ha sido el de estudiar los desafíos actuales para la utilización de este tipo de asistentes en el ámbito de las universidades inteligentes. Asimismo, se ha analizado cómo esta nueva tecnología puede ayudar a los estudiantes en su proceso de aprendizaje y grado de satisfacción. Los resultados de este trabajo se presentan en tres artículos de investigación publicados en revistas científicas indexadas en Web of Science. También se aporta un Registro de la Propiedad Intelectual registrado en el Ministerio de Cultura de España, en la categoría de programa de ordenador, cuyos derechos fueron cedidos a la Universidad de Burgos.Personal voice assistants based on advanced natural language comprehension techniques are shown as a promising resource with regard to the challenge of designing virtual learning platforms. In particular, these resources can support the improvement of the teaching-learning process. The main objective of this work has been to study the current challenges for the use of this type of assistant in the field of smart universities. Likewise, it has been analyzed how this innovative technology can help students in their learning process and their degree of satisfaction. The results of this work are presented in three research articles published in scientific journals indexed on the Web of Science. Also, an Intellectual Property Registry registered with the Ministry of Culture of Spain in the category of computer programs is provided, whose rights were transferred to the University of Burgos

Technology-enhanced Assessment of Thinking Skills in Engineering Sciences

Assessment gilt allgemein als eines der wichtigsten Elemente in der Aus und Weiterbildung. Mit dem Einzug digitaler Technologien in Assessment-Prozesse (auch als E-Assessment bezeichnet), wurden neue Möglichkeiten für personalisierte, unmittelbare und eindrucksvolle Erfahrungen beim Assessment eröffnet. In Zeiten, in denen hochentwickelte, digitale Lernplattformen die Art und Weise verändern, was, wann und wie gelernt werden kann, ist es umso verwunderlicher, wie eingeschränkt die vorhandenen Methoden für die technologie-gestützte Bewertung des Lernens sind. Deutlich wird dies durch die Tatsache, dass aktuelle E-Assessment-Systeme sich größtenteils auf das Replizieren von traditionellen Tests mit Stift und Papier beschränken. Folglich bedarf es neuer Lösungen für die Identifikation, Sammlung, Analyse und Interpretation von Informationen über das individuelle Lernen. Die Berücksichtigung der Anforderungen an das Lernen im 21. Jahrhundert spielt dabei eine entscheidende Rolle. In Erkenntnis dieser Notwendigkeit präsentiert diese Arbeit ein neuartiges Architekturmodell für personalisierte und interaktive E-Assessment-Systeme und -Werkzeuge. Es erlaubt die Integration und Nutzung von interaktiven und immersiven Werkzeugen (z.B. Simulationen oder Animationen) innerhalb von Fragen und Tests, und ermöglicht diesen, sich an die individuellen Charakteristiken der Prüflinge (z.B. Vorwissen, Kontext und Vorlieben) anzupassen. Während das erste Hauptmerkmal (Didaktische Interaktivität) der Annahme gerecht wird, dass Lernen ein Ergebnis von Interaktionen sowie der aktiven Auseinandersetzung mit der jeweiligen Thematik ist, adressiert das zweite Hauptmerkmal (Personalisierung) die bei vielen E-Assessment-Systemen vorherrschende one-size-fits-all Strategie. Die Arbeit beschreibt die Struktur der grundlegenden Komponenten des Architekturmodells. Ein konsistentes Nutzermodell, ein generisches Domänenmodell sowie ein flexibles Adaptionsmodell bilden den zentralen Kern des Gesamtmodells und repräsentieren die Basis für das adaptive Verhalten. Komplettiert wird das Architekturmodell durch eine Komponente für die Modellierung von Fragen sowie einer Komponente für die Durchführung der spezifiierten Adaptionen. Darüber hinaus präsentiert die Arbeit die Implementierung des Architekturmodells durch das webbasierte E-Assessment-System askMe! sowie deren Erprobung und Evaluation nach pädagogischen (Lernunterstützung) sowie technischen (Gebrauchstauglichkeit und Nutzungserlebnis) Gesichtspunkten. Die Ergebnisse dieser Arbeit eröffnen neue Möglichkeiten für zukunftsweisende (E-Assessment-)Systeme, welche in der Lage sind, die Bedürfnisse und Charakteristiken Einzelner zu berücksichtigen sowie mehr Kreativität bei der Beantwortung durch Interaktion mit digitalen Werkzeugen ermöglichen.Assessment is generally recognized as one of the most important elements of an educational experience. Since digital technologies found their way into assessment processes (referred to as technology-enhanced assessment or eassessment), newpossibilities for more personalized, immediate and engaging assessment experiences were opened up. However, especially in current times when sophisticated digital learning environments, mostly enriched by multimedia, virtual/augmented reality technologies, change the way what can be learned, when and how, the methods of assessing students’ learning that have so far been developed are surprisingly limited. This can be demonstrated by the fact that current e-assessment practices simply imitate or replicate traditional pen-and-paper assessments. Consequently, new solutions are needed to identify, gather, analyze and interpret information about students’ learning, especially considering the requirements of the 21st century. In recognizing this need, this thesis proposes a novel architectural model for personalized and interactive e-assessment systems and tools. It allows integrating and using interactive and immersive tools (e.g., simulations or animations) into questions and tests, and enables tailoring them to students’ individual characteristics (e.g., prior knowledge, context and preferences). While the former key feature (didactic interactivity) takes into account the assumption that learning is the result of interaction and the active engagement with the subject matter, respectively, the latter one (personalization) tackles the one-size-fits-all approach mostly applied in traditional e-assessment settings. Furthermore, the thesis describes the structure and the constituent components of the architectural model. A consistent user model, a generic domain model and a flexible adaptation model build up the central part of the overall model and represent the fundamental basis for the adaptive behavior. Each model is managed by an own component and has well-defined interfaces to each other. Additionally, the architectural model is complemented by a question modeling component responsible for representing (interactive) questions, responses, etc. and finally, an adaptive testing engine component that performs the actual adaptations. Moreover, this thesis presents the implementation of the architectural model by the web-based e-assessment system askMe!. It also describes how this system was trialed and evaluated from a pedagogical (learning support) and technical (usability and user experience) point of view. The research and development performed in this thesis open up new opportunities for advanced e-assessment systems, which are able to consider the needs and characteristics of students and allow for more creativity in answering by interacting with digital tools in a variety of ways

Immersive Storymaking : An Emerging Art Form for Virtual Reality Learning Environments

Virtual Reality (VR) technologies are moving stories from non-participatory to participatory formats through the effects of immersion, interaction, presence and agency. The study’s thesis therefore is that the shifting storytelling paradigm from teller-listener to participant-builder requires a new art form that includes a grammar of making. The study presents the concept of storymaking as this emerging art form. The first objective was to develop a conceptual framework for a storymaking practice, and secondly to implement this framework to determine if it satisfied the need for a new paradigm. To do this, the works of key narrative theorists such as Aristotle (2016) and interactive narrative practitioners such as Louchart and Aylett (2005) were referenced in defining a story and making grammar that resulted in the Storymaking Framework. This Framework was then applied to the creation of a 360° VR experience on traditional Trinidad Carnival characters. The method of autoethnography captured important lessons from this practical exercise through researcher-as-instrument. However, to determine the relevance of storymaking, a small group of nine participants (6 students pursuing Caribbean Studies, 2 administrators and 1 teacher) evaluated the experience. The results from pre- and post-experience surveys revealed that the experience did in fact contribute to significant levels of immersion, interaction and presence in participants. Eight participants felt immersed; all nine participants felt sure they were moving through the experience on their own and all participants indicated that they had learned something new from being a part of the narrative. This study therefore pinpointed that the key factor that justified the need for the new storymaking art form was the embodiment of the participant in the story. The inclusion of a participant moved a story from artefact to process, facilitated by narrative and technical elements for an emotional connection to the story and memory making

Eliciting User Experience Information in Early Design Phases. The CARE Approach to In-Vehicle UX

Experience-rich input in early phases of a design process can offer valuable information and inspiration to designers. However, there are methodological challenges linked with efforts to understand future user experiences. Experience encompasses multi-layered and tacit data, such as emotions and value, that are important for commercial success but are difficult to elicit from users for existing products, and even more so for concepts in early design phases. At early design phases, the inevitably incomplete representations of product and use context influences the outcomes. It is typically easier to elicit usability-related aspects, meaning that other aspects of experience may be insufficiently addressed. The contribution of this thesis is an approach for eliciting rich user experience (UX) data in early design phases, building on six studies. This thesis employs in-vehicle user experience as a study case, but results are however presented on a methodological level that can also be of use to other interactive products. The overall research questions are: What signifies in-vehicle UX? How can UX data be elicited for input to novel in-vehicle concepts in early design phases?Firstly, the analysis phase of the design process was addressed, where a multi-method approach was employed to study current in-vehicle UX. UX is an umbrella term that has proven difficult to describe and conceptualise in studies. Therefore, the aim of the first study was to better understand what signifies the specific case of in-vehicle UX. Secondly, how to approach and understand user expectations on future autonomous cars was address in the two following studies, in order to address prospective research of novel systems. A method addressing research on user expectations was developed – Setting the Stage for Autonomous Cars. Thirdly, ideation was addressed in a series of workshops, containing generative and creative efforts for ideating future interactive in-vehicle systems. Methods such as enactment, small-scale scenarios, Wizard of Oz, a lo-fi driving simulator and the developed Setting the Stage for Autonomous Cars method were used. The final studies address concept evaluation, and comparatively explore the effects of choosing different product representations (storyboard and interactive prototype) and study contexts (Virtual Reality and in the field) in early UX evaluation. Based on the outcomes of the studies, an approach is proposed – the CARE approach – for enabling richer and more in-depth UX data in early design phases. This approach suggests that there is a need to Contextualise the researched experience (conveying the intended use situation and sentisising the participants to experience), enabling the participant to Act (enabling interaction even at the stages of very lo-fi concepts), supporting Reflection on the experience (enhanced by generative elements in the methods, such as drawing concepts and enacting use) and enabling the participant to Express the experience (in more ways than by just relying on words). Furthermore, the thesis presents findings regarding what signifies in-vehicle UX, for example whole-body, multi-sensory interactions, the importance of the temporal stage of use, the social and multi-device context, and the changing relationship between user and car with increased automation. The results emphasise the importance of addressing the multisensory use situation in each design phase and for participants to express experiences, not only in words but also through enactment and generative techniques

Multimedia Development of English Vocabulary Learning in Primary School

In this paper, we describe a prototype of web-based intelligent handwriting education system for autonomous learning of Bengali characters. Bengali language is used by more than 211 million people of India and Bangladesh. Due to the socio-economical limitation, all of the population does not have the chance to go to school. This research project was aimed to develop an intelligent Bengali handwriting education system. As an intelligent tutor, the system can automatically check the handwriting errors, such as stroke production errors, stroke sequence errors, stroke relationship errors and immediately provide a feedback to the students to correct themselves. Our proposed system can be accessed from smartphone or iPhone that allows students to do practice their Bengali handwriting at anytime and anywhere. Bengali is a multi-stroke input characters with extremely long cursive shaped where it has stroke order variability and stroke direction variability. Due to this structural limitation, recognition speed is a crucial issue to apply traditional online handwriting recognition algorithm for Bengali language learning. In this work, we have adopted hierarchical recognition approach to improve the recognition speed that makes our system adaptable for web-based language learning. We applied writing speed free recognition methodology together with hierarchical recognition algorithm. It ensured the learning of all aged population, especially for children and older national. The experimental results showed that our proposed hierarchical recognition algorithm can provide higher accuracy than traditional multi-stroke recognition algorithm with more writing variability