Scoping analytical usability evaluation methods: A case study

Analytical usability evaluation methods (UEMs) can complement empirical evaluation of systems: for example, they can often be used earlier in design and can provide accounts of why users might experience difficulties, as well as what those difficulties are. However, their properties and value are only partially understood. One way to improve our understanding is by detailed comparisons using a single interface or system as a target for evaluation, but we need to look deeper than simple problem counts: we need to consider what kinds of accounts each UEM offers, and why. Here, we report on a detailed comparison of eight analytical UEMs. These eight methods were applied to it robotic arm interface, and the findings were systematically compared against video data of the arm ill use. The usability issues that were identified could be grouped into five categories: system design, user misconceptions, conceptual fit between user and system, physical issues, and contextual ones. Other possible categories such as User experience did not emerge in this particular study. With the exception of Heuristic Evaluation, which supported a range of insights, each analytical method was found to focus attention on just one or two categories of issues. Two of the three "home-grown" methods (Evaluating Multimodal Usability and Concept-based Analysis of Surface and Structural Misfits) were found to occupy particular niches in the space, whereas the third (Programmable User Modeling) did not. This approach has identified commonalities and contrasts between methods and provided accounts of why a particular method yielded the insights it did. Rather than considering measures such as problem count or thoroughness, this approach has yielded insights into the scope of each method

A Grounded Theoretical and Linguistic Analysis Approach for Non-Functional Requirements Analysis

An important aspect of the requirements engineering process is the specification of traceable, unambiguous and operationalizable non-functional requirements. This remains a non-trivial task due to the lack of well-documented, systematic procedures that facilitate a structured analysis of the qualitative data that is typically the input to this activity. This research investigates the development of a procedural approach that can potentially fill this gap by incorporating procedural perspectives from Grounded Theory Method, Linguistic Analysis and the Non-Functional Requirement Framework, without significantly deviating from existing practice. This paper describes a preliminary version of this procedural approach along with empirical illustrations using data from a redesign initiative of a library website of a public university in the United States. The paper concludes with a preliminary assessment of the approach and a discussion of the contributions of the research the research

Developing Strategic Reports for National Co-Operative of Malaysia (Angkasa) Using Data Warehouse and Decision Tree Model

Managing an organization requires access to information in order to monitor activities and assess performance. Business Intelligence (BI) solutions provide organizations with timley, itegrated information that is crucial to the understanding of their business. Data Warehouse (DW) technology is one of the important strategic management approaches for decision making in an organizations. The BI combines architectures, tools, databases, analytical tools, and methodologies to enable the implementation of interactive information in generating analytical reports. Strategic reports, which influence the enduring way of the whole company, are typically used by top managers. These kinds of decisions are repeatedly complex and the outcomes unsure, because existing information is habitually incomplete. Managers at this point must normally depend on history experiences and their instincts when making strategic decisions. DW is a technology allows integrating and transforming enterprise data for strategic decision making. Furthermore, Decision Tree (DT) is a decision support tool that uses a tree-like graphof decisions and their possible consequences, including chance event outcomes, resource costs, and utility. The organization, which is, responsible to manage people activities need strategic decisions making. This paper will be focused how to design and develop Strategic Reports using DW and DT Model for National Co-operative Organization of Malaysia (ANGKASA) called DSRNCO, as a case study. This system has been evaluated through the system user feedback by using Computer System Usability Questionnaire (CSUQ), which measures system usability and user satisfaction

Applying cognitive science to digital human modelling for user centred design

To build software which, at the press of a button, can tell you what cognition related hazards there are within an environment or a task, is probably well into the future if it is possible at all. However, incorporating existing tools such as task analysis tools, interface design guidelines and information about general cognitive limitations in humans, could allow for greater evaluative options for cognitive ergonomics. The paper discusses previous approaches to the subject and suggests adding design and evaluative guiding in Digital Human Modelling that will help a user with little or no knowledge of cognitive science to design and evaluate a human- product interaction scenario

Visual analytics for supply network management: system design and evaluation

We propose a visual analytic system to augment and enhance decision-making processes of supply chain managers. Several design requirements drive the development of our integrated architecture and lead to three primary capabilities of our system prototype. First, a visual analytic system must integrate various relevant views and perspectives that highlight different structural aspects of a supply network. Second, the system must deliver required information on-demand and update the visual representation via user-initiated interactions. Third, the system must provide both descriptive and predictive analytic functions for managers to gain contingency intelligence. Based on these capabilities we implement an interactive web-based visual analytic system. Our system enables managers to interactively apply visual encodings based on different node and edge attributes to facilitate mental map matching between abstract attributes and visual elements. Grounded in cognitive fit theory, we demonstrate that an interactive visual system that dynamically adjusts visual representations to the decision environment can significantly enhance decision-making processes in a supply network setting. We conduct multi-stage evaluation sessions with prototypical users that collectively confirm the value of our system. Our results indicate a positive reaction to our system. We conclude with implications and future research opportunities.The authors would like to thank the participants of the 2015 Businessvis Workshop at IEEE VIS, Prof. Benoit Montreuil, and Dr. Driss Hakimi for their valuable feedback on an earlier version of the software; Prof. Manpreet Hora for assisting with and Georgia Tech graduate students for participating in the evaluation sessions; and the two anonymous reviewers for their detailed comments and suggestions. The study was in part supported by the Tennenbaum Institute at Georgia Tech Award # K9305. (K9305 - Tennenbaum Institute at Georgia Tech Award)Accepted manuscrip

A Tool-based Semantic Framework for Security Requirements Specification

Attaining high quality in security requirements specification requires first-rate professional expertise, which is scarce. In fact, most organisations do not include core security experts in their software team. This scenario motivates the need for adequate tool support for security requirements specification so that the human requirements analyst can be assisted to specify security requirements of acceptable quality with minimum effort. This paper presents a tool-based semantic framework that uses ontology and requirements boilerplates to facilitate the formulation and specification of security requirements. A two-phased evaluation of the semantic framework suggests that it is usable, leads to reduction of effort, aids the quick discovery of hidden security threats, and improves the quality of security requirements

Process-oriented evaluation of user interactions in integrated system analysis tools

When computer-based tools are used for analysis of complex systems, the design of user interactions and interfaces becomes an essential part of development that determines the overall quality. The objective of this study is to investigate the processes and results of user interactions with integrated analysis tools to synthesize design implications for future tool development. In this study, two space exploration logistics tools are compared in a controlled user experiment. Through a comparative usability analysis, this study evaluated user performance and perception to provide design implications for future integrated analysis tools. For a comprehensive evaluation, multiple methods were used for data collection, including observation, questionnaire and interview. In addition to a result-oriented performance analysis, a process-oriented approach was used for analyzing patterns in user behaviors and errors. Results are presented with reference to the related features embedded in the interfaces of the two tools. Based on the comparative results, synthesized design insights for hierarchical structure, model transparency, automation, and visualization and feedback are discussed for integrated analysis tools in general.American Society for Engineering Education. National Defense Science and Engineering Graduate Fellowshi

Incorporating cognitive aspects in digital human modelling

To build software which, at the press of a button, can tell you what cognition related hazards there are within an environment or a task, is probably well into the future if it is possible at all. However, incorporating existing tools such as task analysis tools, interface design guidelines and information about general cognitive limitations in humans, could allow for greater evaluative options for cognitive ergonomics. The paper will discuss previous approaches on the subject and suggest adding design and evaluative guiding in DHM that will help a user with little to no knowledge of cognitive science, design and evaluate a human-product interaction scenario