Mobile Application Usability: Heuristic Evaluation and Evaluation of Heuristics

Ger Joyce, Mariana Lilley, Trevor Barker, and Amanda Jefferies, 'Mobile Application Usability: Heuristic Evaluation and Evaluation of Heuristics', paper presented at AHFE 2016 International Conference on Human Factors, Software, and Systems Engineering. Walt Disney World, Florida USA, 27-31 July 2016Many traditional usability evaluation methods do not consider mobile-specific issues. This can result in mobile applications that abound in usability issues. We empirically evaluate three sets of usability heuristics for use with mobile applications, including a set defined by the authors. While the set of heuristics defined by the authors surface more usability issues in a mobile application than other sets of heuristics, improvements to the set can be made

Incorporating Heuristic Evaluation (HE) in the Evaluation of Visual Design of the Eco-Tourism Smartphone App

Heuristic Evaluation (HE) has proven to be important in the development of different computer systems but has not been incorporated in the development of eco-tourism smartphone applications. This results inusability issues that significantly affect user experience (UX) as discussed in literature. This study reports the HE in the design and development of Niranur Agro Farm (NAF) eco-tourism smartphone applications, which could improve UX. Eight experts participated in this study, utilizing the SMART mobile usability heuristic developed for mobile application and the severity rating scale to determine usability issues. The HE findings indicated that 22 usability issues were identified. One issue was rated 4 (catastrophe), four issues were rated 3 (major problem), twelve issues were rated 2 (minor problem) and five issues were rated 1 (cosmetics). Although there are issues rated as 4 and 3, the majority of the issues were considered to be minor (1 and 2 on the scale). Results indicated that it is crucial to incorporate HE into the design and development of the eco-tourism smartphone app to minimize the usability issues faced by users. It further validated that utilizing a specific heuristic for smartphone applications would ensure that all usability issues are correctly categorized and remedied

Experts evaluation of usability for digital solutions directed at older adults: a scoping review of reviews

Background: it is important to standardize the evaluation and reporting procedures across usability studies to guide researchers, facilitate comparisons, and promote high-quality studies. A first step to standardizing is to have an overview of how experts-based usability evaluation studies are reported across the literature. Objectives: to describe and synthesize the procedures of usability evaluation by experts that are being reported to conduct inspection usability assessments of digital solutions relevant for older adults. Methods: a scoping review of reviews was performed using a five-stage methodology to identify and describe relevant literature published between 2009 and 2020 as follows: i) identification of the research question; ii) identification of relevant studies; iii) select studies for review; iv) charting of data from selected literature; and v) collation, summary, and report of results. The research was conducted on five electronic databases: PubMed, ACM Digital Library, IEEE, Scopus, and Web of Science. The articles that met the inclusion criteria were identified, and data extracted for further analysis, including evaluators, current usability inspection methods, and instruments to support usability inspection methods. Results: a total of 3958 articles were identified. After a detailed screening, 12 reviews matched the eligibility criteria. Conclusion: overall, we found a variety of unstandardized procedures and a lack of detail on some important aspects of the assessment, including a thorough description of the evaluators and of the instruments used to facilitate the inspection evaluation such as heuristics checklists. These findings suggest the need for a consensus framework on the experts’ assessment of usability that informs researchers and allows standardization of procedures.in publicatio

Mobile application usability heuristics: Decoupling context-of-use

© 2017 Springer International Publishing AG. All rights are reserved by the Publisher. This is the accepted manuscript version of a conference paper which has been published in final form at ttps://doi.org/10.1007/978-3-319-58634-2_30Context-of-use is a vital consideration when evaluating the usability of mobile applications. Thus, when defining sets of heuristics for the usability evaluation of mobile applications, a common practice has been to include one or more heuristics that consider context-of-use. Yet, most evaluations are conducted within usability labs. Consequently, the aim of this research is to question the utility of attempting to include inherently complex areas of context-of-use within limited sets of mobile application usability heuristics. To address this, a mapping study uncovered six sets of heuristics that can be applied to mobile application usability evaluations. A within-subjects empirical test with six Human-Computer Interaction practitioners evaluated a well-known travel mobile application using three sets of the mapped heuristics. The study found that the common practice of including context-of-use within mobile application usability heuristics is an ineffective approach

Three case studies on methods of working with older people in the design of new technologies

The proportion of older adults in the population is rapidly increasing and the proportion of younger adults to care for them is decreasing. Part of the solution to support older adults in living independently is to provide them with appropriate assistive technologies. To develop technologies that are effective for older adults we need methodologies that are appropriate for working with this user group. Yet there is little systematic research on how to work with older adults and how to adapt methods already used with younger adults. This paper reports on three case studies which investigated the use focus groups, expert evaluations and user evaluations with older adults. In the case of focus groups, the size of the focus group was investigated; for expert evaluations, an existing set of heuristics for evaluating apps for older adults was investigated; for user evaluations, a low-fidelity prototype design was evaluated using think-aloud protocol

Heuristic Evaluation of Play4Fit Health and Fitness App: A Comparison Between Experts and Novices Evaluators

Heuristic evaluation (HE) can be used to effectively identify usability issues in various interfaces. However, it has not been widely used in evaluating smartphone apps, especially in the health and fitness domain. One reason is the lack of HCI experts, which makes incorporating HE into the design process difficult. This paper presents the results of a study that compared HE performed by three HCI experts and three novices in evaluating a gamification app for health and fitness on a smartphone. The study used Smartphone Mobile Application heuRisTics (SMART), which focuses on smartphone apps, and a severity rating scale to determine the severity of the usability issues. These issues were mapped to the SMART heuristic. The findings indicate that novices may identify usability issues that the experts overlooked. While the experts identified eighteen usability issues, the novices found only four; however, the novice’s findings may be used as a substitute for HE when experts are unavailable. Both experts and novices identified two similar usability issues, but their severity ratings differed. One possible solution to address the lack of usability issues identified by novices in HE is to use more novices instead of experts in the evaluation process

Heuristic Evaluation of Play4Fit Health and Fitness App: A Comparison Between Experts and Novices Evaluators

Heuristic evaluation (HE) can be used to effectively identify usability issues in various interfaces. However, it has not been widely used in evaluating smartphone apps, especially in the health and fitness domain. One reason is the lack of HCI experts, which makes incorporating HE into the design process difficult. This paper presents the results of a study that compared HE performed by three HCI experts and three novices in evaluating a gamification app for health and fitness on a smartphone. The study used Smartphone Mobile Application heuRisTics (SMART), which focuses on smartphone apps, and a severity rating scale to determine the severity of the usability issues. These issues were mapped to the SMART heuristic. The findings indicate that novices may identify usability issues that the experts overlooked. While the experts identified eighteen usability issues, the novices found only four; however, the novice’s findings may be used as a substitute for HE when experts are unavailable. Both experts and novices identified two similar usability issues, but their severity ratings differed. One possible solution to address the lack of usability issues identified by novices in HE is to use more novices instead of experts in the evaluation process

App chronic disease checklist: Protocol to evaluate mobile apps for chronic disease self-management

Background: The availability of mobile health apps for self-care continues to increase. While little evidence of their clinical impact has been published, there is general agreement among health authorities and authors that consumers’ use of health apps assist in self-management and potentially clinical decision making. A consumer’s sustained engagement with a health app is dependent on the usability and functionality of the app. While numerous studies have attempted to evaluate health apps, there is a paucity of published methods that adequately recognize client experiences in the academic evaluation of apps for chronic conditions. Objective: This paper reports (1) a protocol to shortlist health apps for academic evaluation, (2) synthesis of a checklist to screen health apps for quality and reliability, and (3) a proposed method to theoretically evaluate usability of health apps, with a view towards identifying one or more apps suitable for clinical assessment. Methods: A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram was developed to guide the selection of the apps to be assessed. The screening checklist was thematically synthesized with reference to recurring constructs in published checklists and related materials for the assessment of health apps. The checklist was evaluated by the authors for face and construct validity. The proposed method for evaluation of health apps required the design of procedures for raters of apps, dummy data entry to test the apps, and analysis of raters’ scores. Results: The PRISMA flow diagram comprises 5 steps: filtering of duplicate apps; eliminating non-English apps; removing apps requiring purchase, filtering apps not updated within the past year; and separation of apps into their core functionality. The screening checklist to evaluate the selected apps was named the App Chronic Disease Checklist, and comprises 4 sections with 6 questions in each section. The validity check verified classification of, and ambiguity in, wording of questions within constructs. The proposed method to evaluate shortlisted and downloaded apps comprises instructions to attempt set-up of a dummy user profile, and dummy data entry to represent in-range and out-of-range clinical measures simulating a range of user behaviors. A minimum score of 80% by consensus (using the Intraclass Correlation Coefficient) between raters is proposed to identify apps suitable for clinical trials. onclusions: The flow diagram allows researchers to shortlist health apps that are potentially suitable for formal evaluation. The evaluation checklist enables quantitative comparison of shortlisted apps based on constructs reported in the literature. The use of multiple raters, and comparison of their scores, is proposed to manage inherent subjectivity in assessing user experiences. Initial trial of the combined protocol is planned for apps pertaining to the self-monitoring of asthma; these results will be reported elsewhere

Establishing usability heuristics for heuristics evaluation in a specific domain: is there a consensus?

Heuristics evaluation is frequently employed to evaluate usability. While general heuristics are suitable to evaluate most user interfaces, there is still a need to establish heuristics for specific domains to ensure that their specific usability issues are identified. This paper presents a comprehensive review of 70 studies related to usability heuristics for specific domains. The aim of this paper is to review the processes that were applied to establish heuristics in specific domains and identify gaps in order to provide recommendations for future research and area of improvements. The most urgent issue found is the deficiency of validation effort following heuristics proposition and the lack of robustness and rigour of validation method adopted. Whether domain specific heuristics perform better or worse than general ones is inconclusive due to lack of validation quality and clarity on how to assess the effectiveness of heuristics for specific domains. The lack of validation quality also affects effort in improving existing heuristics for specific domain as their weaknesses are not addressed