Heuristics Considering UX and Quality Criteria for Heuristics

Heuristic evaluation is a cheap tool with which one can take qualitative measures of a product’s usability. However, since the methodology was first presented, the User Experience (UX) has become more popular but the heuristics have remained the same. In this paper, we analyse the current state of heuristic evaluation in terms of heuristics for measuring the UX. To do so, we carried out a literature review. In addition, we had a look at different heuristics and mapped them with the UX dimensions of the User Experience Questionnaire (UEQ). Moreover, we proposed a quality model for heuristic evaluation and a list of quality criteria for heuristics

A Process for Designing and Developing Interactive Learning Objects for Organisations

Organisations are increasingly adopting e-learning environments for employee and customer training projects. These environments use educational technologies and interactive learning objects (ILOs) to increase the quality of digital training experiences. ILOs are growing in popularity because material is multimodal but even more so, interactive which engages trainees and increases motivation to learn. Several studies have proposed processes and guidelines for improving the usability and user experience (UX) of system applications and websites. However, the processes and guidelines for designing ILOs are limited and the majority of these focus on ILOs adopted in educational institutions. Corporations need to consider additional factors, such as the organisational culture and the disparate profile of users when designing their e-learning environment and the ILOs to be adopted. This paper seeks to answer the research question, “What process should be followed when designing interactive learning objects (ILOs) in a corporate context?”. The purpose of this paper is to propose a process for designing and developing ILOs (PDILO) that can be used to increase the UX of these ILOs and ultimately, the usage thereof in an e-learning environment for corporate contexts. A case study approach was adopted and the case was a software development company incorporating e-learning into its management strategy. The design process was used to design ILOs that formed part of a module in the company’s new e- learning environment. The proposed process can be used to guide content developers with the design of ILOs in e-learning environments and ultimately improve the success thereof

A bi-objective genetic algorithm approach to risk mitigation in project scheduling

A problem of risk mitigation in project scheduling is formulated as a bi-objective optimization problem, where the expected makespan and the expected total cost are both to be minimized. The expected total cost is the sum of four cost components: overhead cost, activity execution cost, cost of reducing risks and penalty cost for tardiness. Risks for activities are predefined. For each risk at an activity, various levels are defined, which correspond to the results of different preventive measures. Only those risks with a probable impact on the duration of the related activity are considered here. Impacts of risks are not only accounted for through the expected makespan but are also translated into cost and thus have an impact on the expected total cost. An MIP model and a heuristic solution approach based on genetic algorithms (GAs) is proposed. The experiments conducted indicate that GAs provide a fast and effective solution approach to the problem. For smaller problems, the results obtained by the GA are very good. For larger problems, there is room for improvement

Developing human-centred framework and guidelines for disc golf application design

Abstract. Disc golf is an increasingly popular sport, especially in Finland and the USA. The disc golf industry and Professional Disc Golf Association have proceeded in the digital transformation; therefore, disc golf applications have penetrated the markets. There is a need for evaluating and developing the current quality of disc golf application design and looking for general guidelines for designing comprehensive applications. This thesis aims to study what kind of framework is hidden behind the disc golf application design from a human-centred perspective when also several stakeholders are concerned. The study is based on theoretical and empirical research. Theoretical research is conducted by literary review of existing disc golf application design guidelines, together with human-centred design, user-centred design, and stakeholder-centred design theory and methodologies in general. Nevertheless, a research gap is found as far as disc golf, golf, and sports application design are concerned. Thus, the review expands to study field practitioners and mobile health context. The literary review also examines disc golf as a sport and in numbers, together with current disc golf applications. Empirical research adapts human-centred design methods by studying users’ expectations, needs, and wishes in the mean of questionnaires, and interviews. Also, the context of use is identified and visited, user requirements are mapped, disc golf personas are created, and the disc golf application’s stakeholders are identified. Then, the method adapted from service-dominant logic is used for preparing value proposals for all disc golf application users, and other stakeholders. By adapting the design guidelines from the literature and empirical findings, the human-centred guidelines for disc golf application design (HGDG) framework is presented as the first main finding of the thesis. The HGDG framework can be used for disc golf application design and ensuring the usage of the human-centred approach. The second main finding, 15 design guidelines and instructions for disc golf application design reinforces the HGDG framework. The third main finding suggests using 15 design guidelines as a metric for evaluating if the designed application achieves an ideal win-win-…-win situation between of application provider, users, and other stakeholders

Heuristic usability evaluation on games: a modular approach

Heuristic evaluation is the preferred method to assess usability in games when experts conduct this evaluation. Many heuristics guidelines have been proposed attending to specificities of games but they only focus on specific subsets of games or platforms. In fact, to date the most used guideline to evaluate games usability is still Nielsen’s proposal, which is focused on generic software. As a result, most evaluations do not cover important aspects in games such as mobility, multiplayer interactions, enjoyability and playability, etc. To promote the usage of new heuristics adapted to different game and platform aspects we propose a modular approach based on the classification of existing game heuristics using metadata and a tool, MUSE (Meta-heUristics uSability Evaluation tool) for games, which allows a rebuild of heuristic guidelines based on metadata selection in order to obtain a customized list for every real evaluation case. The usage of these new rebuilt heuristic guidelines allows an explicit attendance to a wide range of usability aspects in games and a better detection of usability issues. We preliminarily evaluate MUSE with an analysis of two different games, using both the Nielsen’s heuristics and the customized heuristic lists generated by our tool.Unión Europea PI055-15/E0

Facilitating heuristic evaluation for novice evaluators

Heuristic evaluation (HE) is one of the most widely used usability evaluation methods. The reason for its popularity is that it is a discount method, meaning that it does not require substantial time or resources, and it is simple, as evaluators can evaluate a system guided by a set of usability heuristics. Despite its simplicity, a major problem with HE is that there is a significant gap in the quality of results produced by expert and novice evaluators. This gap has made some scholars question the usefulness of the method as they claim that the evaluation results are a product of the evaluator’s experience rather than the method itself. In response, the goal of this thesis is to bridge the gap between expert and novice evaluators. Based on interviews with 15 usability experts, which focused on their experience with the method, the difficulties they faced when they were novices, and how they overcame such difficulties, it presents a comprehensive protocol called Coherent Heuristic Evaluation (CoHE). This step-by-step protocol guides novice evaluators from the moment they decide to conduct an evaluation until the submission of their evaluation report. This protocol was verified by conducting an experiment to observe the difference between novices using the CoHE protocol and novices using Nielsen’s 10 usability heuristics without the guidance. The experiment involved 20 novices performing two sessions; the first was an understanding session where the novices read and understood the heuristics and the second was an inspecting session where they inspected a system. The findings show that, while evaluators take more time to read and evaluate a system using CoHE, they tend to identify more problems. The experiment also demonstrated that CoHE can improve the thoroughness, effectiveness, and f-measure of evaluation. However, the validity of CoHE was comparable to that of HE

Comparison of UX evaluation methods through evaluation of UX of the prototype of a matching platform for the rental housing market in Finland – Sopia.

Key concepts of this Master’s thesis are user experience (UX), usability, startup and UX evaluation methods. The research question is how startups should evaluate their website’s UX. To answer it I conducted a study of five UX evaluation methods: heuristic evaluation (HE), cognitive walkthrough (CW), tree testing, system usability scale (SUS), brainstorming through theoretical and empirical analysis. I collected empirical data in two ways. First, I interviewed seven UX field practitioners on their experiences of different UX evaluation methods. Second, I applied the evaluation methods to the website prototype of a digital startup called Sopia. To be able to consistently compare the UX evaluation methods, I created a theory-based framework that includes a set of generic parameters describing evaluation methods, and the constraints of the startup. Based on my findings three UX evaluation methods would be useful in the startup context: heuristic evaluation, cognitive walkthrough and brainstorming. Practitioners tend to select flexible, fast and simple evaluation methods. Cognitive walkthrough and brainstorming match these criteria. Cognitive walkthrough when conducted with potential end users, reveals UX mistakes at an early stage of UX design. Brainstorming carried out within the design team afterwards helps to find resolutions for the revealed usability problems. Heuristic evaluation should not be carried out in its traditional definition with usability experts. However, startups should learn 10 heuristics as 10 usability principles to create the ground of good UX. The key contribution of my study is the framework of Minimum Viable UX Evaluation Methods for Startups. The framework represents the list of necessary UX evaluation tools that each startup, despite time, money and human resource constraints, should follow. Each evaluation method, based on my findings, is unavoidable to help the startup to progress with product development

Incorporating Heuristic Evaluation (HE) in the Evaluation of Visual Design of the Eco-Tourism Smartphone App

Heuristic Evaluation (HE) has proven to be important in the development of different computer systems but has not been incorporated in the development of eco-tourism smartphone applications. This results inusability issues that significantly affect user experience (UX) as discussed in literature. This study reports the HE in the design and development of Niranur Agro Farm (NAF) eco-tourism smartphone applications, which could improve UX. Eight experts participated in this study, utilizing the SMART mobile usability heuristic developed for mobile application and the severity rating scale to determine usability issues. The HE findings indicated that 22 usability issues were identified. One issue was rated 4 (catastrophe), four issues were rated 3 (major problem), twelve issues were rated 2 (minor problem) and five issues were rated 1 (cosmetics). Although there are issues rated as 4 and 3, the majority of the issues were considered to be minor (1 and 2 on the scale). Results indicated that it is crucial to incorporate HE into the design and development of the eco-tourism smartphone app to minimize the usability issues faced by users. It further validated that utilizing a specific heuristic for smartphone applications would ensure that all usability issues are correctly categorized and remedied

The UX of things: exploring UX principles to inform security and privacy design in the smart home

Smart homes are under attack. Threats can harm both the security of these homes and the privacy of their inhabitants. As a result, in addition to delivering pleasant and aesthetic experiences, smart devices need to protect households from vulnerabilities and attacks. Further, the need for user-centered security and privacy design is particularly important for such an environment, given that inhabitants are demographically-diverse (e.g., age, gender, educational level) and have different skills and (dis)abilities. Prior work has explored different usable security and privacy solutions for smart homes; however, the applicability of user eXperience (UX) principles to security and privacy design is under-explored. This research project aims to address the on-going challenge of security and privacy in the smart home through the lens of UX design. The objective of this thesis is two-fold. First, to investigate how UX factors and principles affect the security and privacy of smart home users. Secondly, to inform product design through the development of an empirically-tested framework for UX design of security and privacy in smart home products. In the first step, we explored the relationship between UX, security, and privacy in smart homes from user and designer perspectives: through (i) conducting a qualitative interview study with smart home users (n=13) and (ii) analyzing an ethnomethodologically informed study of six UK households living in smart homes (n=6); and, we then explored the role of UX in the design of security, privacy and data protection in smart homes through qualitative semi-structured interviews with smart home users, designers and business leaders through two rounds of interviews (n=20, n=20). In the second step, using conceptual framework analysis, we systematically analyzed our previously collected data and the literature to construct a framework of design heuristics for consent and permission in smart homes. We applied these heuristics in four participatory co-design workshops and reported on their use. We further analyzed the use of the heuristics through thematic analysis highlighting how the heuristics were used, their purpose, and their effectiveness. By bringing UX design to the smart home security and privacy table, we believe that this research project will have a significant impact on academia, industry, and government organizations. Our thesis will improve design practices for security and privacy in domestic smart devices while addressing wider challenges, opportunities, and future work

Useful shortcuts: Using design heuristics for consent and permission in smart home devices

Prior research in smart home privacy highlights significant issues with how users understand, permit, and consent to data use. Some of the underlying issues point to unclear data protection regulations, lack of design principles, and dark patterns. In this paper, we explore heuristics (also called “mental shortcuts” or “rules of thumb”) as a means to address security and privacy design challenges in smart homes. First, we systematically analyze an existing body of data on smart homes to derive a set of heuristics for the design of consent and permission. Second, we apply these heuristics in four participatory co-design workshops (n = 14) and report on their use. Third, we analyze the use of the heuristics through thematic analysis highlighting heuristic application, purpose, and effectiveness in successful and unsuccessful design outcomes. We conclude with a discussion of the wider challenges, opportunities, and future work for improving design practices for consent in smart homes