Adapting heuristic evaluation for use in information visualization

Orientador: Celmar Guimarães da SilvaDissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de TecnologiaResumo: A Avaliação Heurística é um método clássico de avaliação de interfaces de um sistema interativo na área de Interação Humano-Computador. Pesquisadores e desenvolvedores de software usam esta técnica com frequência, já que ela é rápida, barata e fácil de ser executada. Porém, para usar a técnica em outros sistemas de domínio específico, é necessário criar um novo conjunto de heurística capaz de identificar os problemas dessa área. Em Visualização de Informação (InfoVis), a técnica é utilizada com o conjunto proposto por Nielsen, que possui apenas heurísticas de usabilidade, deixando de cobrir outros conceitos importantes de InfoVis. Na literatura, estão presentes conjuntos de recomendações que cobrem os conceitos de InfoVis, entretanto, muitas destas recomendações não são apresentadas como heurísticas, ou então são restritas a um determinado contexto. Desta forma, este trabalho apresenta um método para criar um conjunto de heurísticas de InfoVis, para ser usado na Avaliação Heurística. O método agrupa as heurísticas e recomendações encontradas na literatura, e cria novas heurísticas com base em cada grupo formado. Assim, um novo conjunto com 15 heurísticas genéricas foi criado, a partir de um conjunto com 62 heurísticas e recomendações e, posteriormente, avaliado. A hipótese é que o novo conjunto irá auxiliar os avaliadores a considerar um conjunto mais amplo de conceitos de visualização durante a Avaliação Heurística com, possivelmente, menos esforço cognitivo, quando comparado com a aplicação direta das 62 heurísticas e recomendaçõesAbstract: Heuristic evaluation technique is a classical evaluation method of user interface in Human-Computer Interaction area. Researchers and software developers broadly use it, given that it is fast, cheap and easy to use. Using it in other system of specific domains demands creating a new heuristic set able to identify common problems of these areas. Information Visualization (InfoVis) researchers commonly use this technique with the original usability heuristic set proposed by Nielsen, which does not cover many relevant aspects of InfoVis. InfoVis literature presents sets of guidelines that cover InfoVis concepts, but it does not present most of them as heuristics, or they cover much specific context. This work presents a method to define a set of InfoVis heuristics for use in Heuristic Evaluation. The method clusters heuristics and guidelines that was found in the literature, and creates a new heuristic based on each group. Thus, a new set of 15 generic heuristics was created, from a set of 62 heuristics and guidelines, and after it was evaluated. The new set will help evaluators to consider a broad set of visualization aspects during Heuristic Evaluation, with possibly less cognitive effort, when compared with a direct application of the 62 heuristics and guidelinesMestradoSistemas de Informação e ComunicaçãoMestre em TecnologiaCAPE

Combinations of methods for collaborative evaluation of the usability of interactive software systems

Usability is a fundamental quality characteristic for the success of an interactive system. It is a concept that includes a set of metrics and methods in order to obtain easy-to-learn and easy-to-use systems. Usability Evaluation Methods, UEM, are quite diverse; their application depends on variables such as costs, time availability, and human resources. A large number of UEM can be employed to assess interactive software systems, but questions arise when deciding which method and/or combination of methods gives more (relevant) information. We propose Collaborative Usability Evaluation Methods, CUEM, following the principles defined by the Collaboration Engineering. This paper analyzes a set of CUEM conducted on different interactive software systems. It proposes combinations of CUEM that provide more complete and comprehensive information about the usability of interactive software systems than those evaluation methods conducted independentl

Defining usability heuristics for adoption and efficiency of an electronic workflow document management system

Usability heuristics have been established for different uses and applications as general guidelines for user interfaces. These can affect the implementation of industry solutions and play a significant role regarding cost reduction and process efficiency. The area of electronic workflow document management (EWDM) solutions, also known as workflow, lacks a formal definition of usability heuristics. With the advent of new technologies such as mobile devices, defining a set of usability heuristics contributes to the adoption and efficiency of an EWDM system. Workflow usability has been evaluated for various industries. Most significantly research has been done for electronic healthcare records (EHR). In other areas such as the financial sector and educational institutions there is also some literature available but not as abundant as for EHR. This was identified as a possible research limitation. The general purpose of this research was to establish and validate an overarching set of usability heuristics for EWDM in general. This was approached by conducting a literature review and a survey on 32 workflow consultants from Hyland Software, Inc. Quantitative and qualitative data was collected focusing on the study’s main research question: “what usability heuristics should be defined to ensure the adoption and efficiency of a workflow implementation? Findings based on regression testing and expert opinions have suggested a proposed set of usability heuristics. The final list consists of: adaptability to diverse platforms, user control, system feedback, intuitive interfaces, visibility on mobile devices, error management, help, and documentation

Combinations of Methods for Collaborative Evaluation of the Usability of Interactive Software Systems

Usability is a fundamental quality characteristic for the success of an interactive system. It is a concept that includes a set of metrics and methods in order to obtain easy-to-learn and easy-to-use systems. Usability Evaluation Methods, UEM, are quite diverse; their application depends on variables such as costs, time availability, and human resources. A large number of UEM can be employed to assess interactive software systems, but questions arise when deciding which method and/or combination of methods gives more (relevant) information. We propose Collaborative Usability Evaluation Methods, CUEM, following the principles defined by the Collaboration Engineering. This paper analyzes a set of CUEM conducted on different interactive software systems. It proposes combinations of CUEM that provide more complete and comprehensive information about the usability of interactive software systems than those evaluation methods conducted independently

Combinations of Methods for Collaborative Evaluation of the Usability of Interactive Software Systems

Usability is a fundamental quality characteristic for the success of an interactive system. It is a concept that includes a set of metrics and methods in order to obtain easy-to-learn and easy-to-use systems. Usability Evaluation Methods, UEM, are quite diverse; their application depends on variables such as costs, time availability, and human resources. A large number of UEM can be employed to assess interactive software systems, but questions arise when deciding which method and/or combination of methods gives more (relevant) information. We propose Collaborative Usability Evaluation Methods, CUEM, following the principles defined by the Collaboration Engineering. This paper analyzes a set of CUEM conducted on different interactive software systems. It proposes combinations of CUEM that provide more complete and comprehensive information about the usability of interactive software systems than those evaluation methods conducted independently

Método de evaluación de usabilidad aplicada a productos de software que facilitan el acceso a herramientas informáticas de personas con enfermedades que afectan la motricidad

La computación es parte intrínseca de nuestro día a día. No obstante, existen enfermedades que imposibilitan su uso de manera eficiente. Por esto, numerosas organizaciones y centros de investigación desarrollan software para facilitar el acceso al uso de los computadores a las personas que padecen de alguna enfermedad que altera de manera significativa su motricidad. La evaluación de usabilidad del software a menudo se realiza de forma empírica. Esta tesis de maestría tiene por objetivo mostrar un método ponderado para evaluar la usabilidad de software diseñado para facilitar el acceso a la computación a personas con deficiencias motoras usando un enfoque heurístico. La evaluación heurística tiene un gran potencial de adaptabilidad, de manera que puede ser usado para encontrar problemas de usabilidad en estos software. El propósito fue desarrollar un nuevo conjunto de heurísticas que pueden ser utilizados por los diseñadores para evaluar las maquetas y prototipos funcionales.Abstract: The computer is an intrinsic part of our daily life. However, there are conditions that preclude its use efficiently. Thus, many institutions and research centers develop software to facilitate access to computers for people who suffer from a disease that significantly alters their motor skills. The usability evaluation of software is often done empirically. The article aims to show a weighted method for evaluating the usability of software designed to facilitate access to computing for people with mobility impairments using a heuristic approach. Heuristic evaluation has great potential for adaptability, so that can be used to find usability problems in these software. The objective of this research was to develop a new set of heuristics that can be used by designers to evaluate the models and functional prototypes.Maestrí

The synthesis of a unified pedagogy for the design and evaluation of e-learning software for high-school computing

In recent decades, several countries have applied a resurgent focus to high-school computer science in the hope that it will provide the foundations for strong and innovative national IT sectors. The UK is one example undertaking this transition, and experiencing the associated challenges placed on the teaching community. In parallel, recent years have seen a trend towards enriching education with digital resources, specifically e-learning software. This study offers a practical contribution to the computer science teaching community by supporting the provision of e-learning software, and hence the increased use of e-learning in high-school teaching. However, it recognises that there remains a concern over the inconsistent pedagogical quality of many e-learning implementations. To safeguard the pedagogical quality of e-learning software, this study offers a research contribution by defining: (1) a comprehensive set of pedagogical heuristics to inform the design of e-learning software; (2) an associated e-learning evaluation protocol to guide the evaluation and selection of e-learning software for use in schools; and in doing so, (3) contributes to the under-researched area of high- school computing pedagogy. The proposed pedagogy synthesises a vast body of knowledge on learning theories into a comprehensive, yet accessible, set of heuristics. These heuristics supplement existing literature by focusing more tightly and in depth on pedagogy, rather than usability. The pedagogy synthesises the following learning theories: constructivism, social constructivism, connectivism, and cognitive load, and additionally gives pedagogical focus to VARK learning styles, ARCS motivational design, collaborative learning, gamification, and computational thinking. The e-learning evaluation protocol builds upon existing best practice in evaluation procedures but is unique in its characteristics and focus. The study follows a rigorous three phase mixed methods exploratory design in which the e-learning pedagogy and evaluation protocol were explored and iteratively developed in concert with input and evaluation from education experts and teachers. In parallel, practice-based input was secured via student usage of prototype e-learning software designed in adherence to the pedagogy. The findings of this research offer preliminary validation of the appropriateness and comprehensiveness of the e- learning pedagogy, and the final phase demonstrates statistically significant learning increases based on student usage of the e-learning software prototype. Additionally, this research offers preliminary validation of the reliability and validity of the evaluation protocol. Building on the findings of this research, several possibilities are outlined to further empirically establish this research, or develop it further into new avenues