Classification and prioritization of usability problems using an augmented classification scheme

Various methods exist for conducting usability evaluation studies in health care. But although the methodology is clear, no usability evaluation method provides a framework by which the usability reporting activities are fully standardized. Despite the frequent use of forms to report the usability problems and their context-information, this reporting is often hindered by information losses. This is due to the fact that evaluators’ problem descriptions are based on individual judgments of what they find salient about a usability problem at a certain moment in time. Moreover, usability problems are typically classified in terms of their type, number, and severity. These classes are usually devised by the evaluator for the purpose at hand and the used problem types often are not mutually exclusive, complete and distinct. Also the impact of usability problems on the task outcome is usually not taken into account. Consequently, problem descriptions are often vague and even when combined with their classification in type or severity leave room for multiple interpretations when discussed with system designers afterwards. Correct interpretation of these problem descriptions is then highly dependent upon the extent to which the evaluators can retrieve relevant details from memory. To remedy this situation a framework is needed guiding usability evaluators in high quality reporting and unique classification of usability problems. Such a framework should allow the disclosure of the underlying essence of problem causes, the severity rating and the classification of the impact of usability problems on the task outcome. The User Action Framework (UAF) is an existing validated classification framework that allows the unique classification of usability problems, but it does not include a severity rating nor does it contain an assessment of the potential impact of usability flaws on the final task outcomes. We therefore augmented the UAF with a severity rating based on Nielsen’s classification and added a classification for expressing the potential impact of usability problems on final task outcomes. Such an augmented scheme will provide the necessary information to system developers to understand the essence of usability problems, to prioritize problems and to tackle them in a system redesign. To investigate the feasibility of such an augmented scheme, it was applied to the results of usability studies of a computerized physician order entry system (CPOE). The evaluators classified the majority of the usability problems identically by use of the augmented UAF. In addition it helped in differentiating problems that looked similar but yet affect the user–system interaction and the task results differently and vice versa. This work is of value not only for system developers but also for researchers who want to study the results of other usability evaluation studies, because this scheme makes the results of usability studies comparable and easily retrievable

The Web-Based Usability Heuristic Survey Supports User Satisfaction

Competition between companies is now getting stronger strongly related to the company's mechanism in providing the services provided. Companies can build ways that can maintain customer loyalty. The approach can be done with web-based Usability Testing adopting an application user acceptance model, a usability aspect analysis that acts as a customer who enjoys a company's products and services. At present, there are already many theories of user acceptance models for an application. One of them is the user satisfaction model (User Satisfaction). This review specifically also identifies that content content has a significant positive effect on website visitor satisfaction

ارزیابی مشکلات تعاملی کاربران با سیستم اطلاعات فیزیوتراپی

زمینه و هدف:سیستم اطلاعات فیزیوتراپی یکی از سیستمهای اطلاعات بهداشتی است که برای افزایش کارآیی و اثربخشی خدمات این بخش بکار می رود. مطالعات نشان داده اند که تعامل کاربران با برخی از سیستم های اطلاعاتی به دلیل مشکلات کاربردپذیری دشوار است. هدف این مطالعه ارزیابی مشکلات تعاملی کاربران با سیستم اطلاعات فیزیوتراپی می باشد. مواد و روش ها:این مطالعه یک پژوهش توصیفی˗ مقطعی است که در آن با استفاده از روش ارزیابی هیورستیک مشکلات تعاملی کاربران سیستم اطلاعات فیزیوتراپی مورد استفاده در 105 بیمارستان کشور توسط سه ارزیاب شناسایی، طبقه بندی و تعیین شدت شدند. داده ها با استفاده از یک فرم گردآوری طراحی شده در نرم افزار اکسل گردآوری شدند. نتایج: در این مطالعه 82 مشکل تعاملی منفرد شناسایی گردید که بیشترین آنها با 28% (23n=) مربوط به مغایرت طراحی سیستم با اصول رایج در دنیای واقعی با میانگین شدت بزرگ بود. کمترین آنها مربوط به مشکلات عدم وجود قسمت راهنمایی و مستندسازی و جلوگیری از خطا در سیستم به ترتیب با میانگین شدت بزرگ و کوچک بودند. نتیجه گیری: برخی از سیستم هایی که امروزه در بسیاری از مراکز بهداشتی درمانی مورد استفاده قرار می گیرند، از جمله سیستم مورد بررسی در این مطالعه، دارای مشکلات متعددی می باشند که تعامل کاربران با آنها را دچار مشکل کرده و می تواند باعث ایجاد خطا و آسیب به بیماران شوند. برای شناسایی و رفع این گونه مشکلات توصیه می شود ارزیابی کاربردپذیری این سیستم ها به صورت مستمر انجام شود تا باعث افزایش رضایتمندی کاربران، بهبود روندکاری و ارتقای امنیت بیماران شود

A usability study to improve a clinical decision support system for the prescription of antibiotic drugs

Objective A clinical decision support system (CDSS) for empirical antibiotic treatment has the potential to increase appropriate antibiotic use. Before using such a system on a broad scale, it needs to be tailored to the users prefer

How to co-design a prototype of a clinical practice tool: a framework with practical guidance and a case study

Clinical tools for use in practice-such as medicine reconciliation charts, diagnosis support tools and track-and-trigger charts-are endemic in healthcare, but relatively little attention is given to how to optimise their design. User-centred design approaches and co-design principles offer potential for improving usability and acceptability of clinical tools, but limited practical guidance is currently available. We propose a framework (FRamework for co-dESign of Clinical practice tOols or 'FRESCO') offering practical guidance based on user-centred methods and co-design principles, organised in five steps: (1) establish a multidisciplinary advisory group; (2) develop initial drafts of the prototype; (3) conduct think-aloud usability evaluations; (4) test in clinical simulations; (5) generate a final prototype informed by workshops. We applied the framework in a case study to support co-design of a prototype track-and-trigger chart for detecting and responding to possible fetal deterioration during labour. This started with establishing an advisory group of 22 members with varied expertise. Two initial draft prototypes were developed-one based on a version produced by national bodies, and the other with similar content but designed using human factors principles. Think-aloud usability evaluations of these prototypes were conducted with 15 professionals, and the findings used to inform co-design of an improved draft prototype. This was tested with 52 maternity professionals from five maternity units through clinical simulations. Analysis of these simulations and six workshops were used to co-design the final prototype to the point of readiness for large-scale testing. By codifying existing methods and principles into a single framework, FRESCO supported mobilisation of the expertise and ingenuity of diverse stakeholders to co-design a prototype track-and-trigger chart in an area of pressing service need. Subject to further evaluation, the framework has potential for application beyond the area of clinical practice in which it was applied

Assessing the reliability, validity and acceptance of a classification scheme of usability problems (CUP)

Post-print (lokagerð höfunda)The aim of this study was to evaluate the Classification of Usability Problems (CUP) scheme. The goal of CUP is to classify usability problems further to give user interface developers better feedback to improve their understanding of usability problems, help them manage usability maintenance, enable them to find effective fixes for UP, and prevent such problems from reoccurring in the future. First, reliability was evaluated with raters of different levels of expertise and experience in using CUP. Second, acceptability was assessed with a questionnaire. Third, validity was assessed by developers in two field studies. An analytical comparison was also made to three other classification schemes. CUP reliability results indicated that the expertise and experience of raters are critical factors for assessing reliability consistently, especially for the more complex attributes. Validity analysis results showed that tools used by developers must be tailored to their working framework, knowledge and maturity. The acceptability study showed that practitioners are concerned with the effort spent in applying any tool. To understand developers’ work and the implications of this study two theories are presented for understanding and prioritising UP. For applying classification schemes, the implications of this study are that training and context are needed.Peer Reviewe

Classification and prioritization of usability problems using an augmented classification scheme

AbstractVarious methods exist for conducting usability evaluation studies in health care. But although the methodology is clear, no usability evaluation method provides a framework by which the usability reporting activities are fully standardized. Despite the frequent use of forms to report the usability problems and their context-information, this reporting is often hindered by information losses. This is due to the fact that evaluators’ problem descriptions are based on individual judgments of what they find salient about a usability problem at a certain moment in time. Moreover, usability problems are typically classified in terms of their type, number, and severity. These classes are usually devised by the evaluator for the purpose at hand and the used problem types often are not mutually exclusive, complete and distinct. Also the impact of usability problems on the task outcome is usually not taken into account. Consequently, problem descriptions are often vague and even when combined with their classification in type or severity leave room for multiple interpretations when discussed with system designers afterwards. Correct interpretation of these problem descriptions is then highly dependent upon the extent to which the evaluators can retrieve relevant details from memory.To remedy this situation a framework is needed guiding usability evaluators in high quality reporting and unique classification of usability problems. Such a framework should allow the disclosure of the underlying essence of problem causes, the severity rating and the classification of the impact of usability problems on the task outcome. The User Action Framework (UAF) is an existing validated classification framework that allows the unique classification of usability problems, but it does not include a severity rating nor does it contain an assessment of the potential impact of usability flaws on the final task outcomes. We therefore augmented the UAF with a severity rating based on Nielsen’s classification and added a classification for expressing the potential impact of usability problems on final task outcomes. Such an augmented scheme will provide the necessary information to system developers to understand the essence of usability problems, to prioritize problems and to tackle them in a system redesign. To investigate the feasibility of such an augmented scheme, it was applied to the results of usability studies of a computerized physician order entry system (CPOE).The evaluators classified the majority of the usability problems identically by use of the augmented UAF. In addition it helped in differentiating problems that looked similar but yet affect the user–system interaction and the task results differently and vice versa. This work is of value not only for system developers but also for researchers who want to study the results of other usability evaluation studies, because this scheme makes the results of usability studies comparable and easily retrievable

Usability engineering in the design and evaluation of a functional electrical stimulation system for upper limb rehabilitation

Chronic physical impairment of the hemiplegic upper limb (UL) is seen in an estimated 50-70% of stroke patients, who place a high priority on regaining upper limb function. Current therapy is insufficiently intensive, often not task-oriented and hence poorly aligned with the evidence base. Functional electrical stimulation (FES) has the potential to not only increase the intensity of task-focused therapy, but also provide certain unique features, notably direct excitation of lower motor neurons. However, current FES systems are limited in their functionality and/or difficult to use. Systems are also poorly aligned to therapists’ ways of working and uptake remains limited. To address these problems, a novel FES technology (UL FES Rehab Tool) has been developed. The control system design is reported in Sun, (2014). The aims of my thesis were to: 1) design a Graphical User Interface (GUI) that would enable therapists to quickly and easily set up an individually tailored library of FES tasks for each patient; 2) evaluate the usability and functionality of the UL FES Rehab Tool(software and hardware) in both laboratory (lab) and clinical settings. An iterative, mixed methods, five-phase usability engineering approach was used to design and evaluate the UL FES Rehab Tool. Phases one to three incorporated identification of therapists’ requirements, a user ‘assisted walkthrough’ of the software with expert and novice FES users and ‘rapid prototyping’ of the full system, using healthy participants. Further usability testing of the software & hardware was conducted in phase four with 1 physiotherapist and 6 patients, (total of 24 visits), in the chronic stage post-stroke. The work demonstrated in detail, for the first time, the impact of therapist involvement in the design of novel rehabilitation technology. To address therapists’ focus on setup time, using the phase four data set, a novel model to predict setup time was devised. This model was able to explain 51% of the variance in setup time based on two parameters, task complexity and patient impairment. Finally, in phase five, a summative usability evaluation of the final prototype was carried out in 2 sub-acute stroke units. Four therapists and 1 rehabilitation assistant used the UL FES Rehab Tool with 6 patients in the acute stage post-stroke. The UL FES Rehab Tool enabled all therapists and one therapy assistant to effectively deliver FES assisted upper limb task-oriented therapy to a range of stroke patients (Fugl-Meyer scores 8–65). The usability methods effectively captured objective and subjective feedback from therapists and patients. However the previous setup time model was unable to predict setup time, suggesting other factors were important in a clinical setting. Although participant numbers were low, the results suggested therapists’ predisposition to using technology and post-training confidence in using the technology may influence their willingness to engage with novel rehabilitation technologies. This study is the first to describe in detail the impact of a usability engineering approach on the design of a complex upper limb rehabilitation technology from early stage design to clinical evaluation. These methods can be generalised to other studies seeking to explore the usability of new forms of rehabilitation technologies