Usability evaluation of digital libraries: a tutorial

This one-day tutorial is an introduction to usability evaluation for Digital Libraries. In particular, we will introduce Claims Analysis. This approach focuses on the designers’ motivations and reasons for making particular design decisions and examines the effect on the user’s interaction with the system. The general approach, as presented by Carroll and Rosson(1992), has been tailored specifically to the design of digital libraries. Digital libraries are notoriously difficult to design well in terms of their eventual usability. In this tutorial, we will present an overview of usability issues and techniques for digital libraries, and a more detailed account of claims analysis, including two supporting techniques – simple cognitive analysis based on Norman’s ‘action cycle’ and Scenarios and personas. Through a graduated series of worked examples, participants will get hands-on experience of applying this approach to developing more usable digital libraries. This tutorial assumes no prior knowledge of usability evaluation, and is aimed at all those involved in the development and deployment of digital libraries

Making intelligent systems team players: Case studies and design issues. Volume 1: Human-computer interaction design

Initial results are reported from a multi-year, interdisciplinary effort to provide guidance and assistance for designers of intelligent systems and their user interfaces. The objective is to achieve more effective human-computer interaction (HCI) for systems with real time fault management capabilities. Intelligent fault management systems within the NASA were evaluated for insight into the design of systems with complex HCI. Preliminary results include: (1) a description of real time fault management in aerospace domains; (2) recommendations and examples for improving intelligent systems design and user interface design; (3) identification of issues requiring further research; and (4) recommendations for a development methodology integrating HCI design into intelligent system design

Improving fairness in machine learning systems: What do industry practitioners need?

The potential for machine learning (ML) systems to amplify social inequities and unfairness is receiving increasing popular and academic attention. A surge of recent work has focused on the development of algorithmic tools to assess and mitigate such unfairness. If these tools are to have a positive impact on industry practice, however, it is crucial that their design be informed by an understanding of real-world needs. Through 35 semi-structured interviews and an anonymous survey of 267 ML practitioners, we conduct the first systematic investigation of commercial product teams' challenges and needs for support in developing fairer ML systems. We identify areas of alignment and disconnect between the challenges faced by industry practitioners and solutions proposed in the fair ML research literature. Based on these findings, we highlight directions for future ML and HCI research that will better address industry practitioners' needs.Comment: To appear in the 2019 ACM CHI Conference on Human Factors in Computing Systems (CHI 2019

Entangled Design Knowledge: Relationships as an Approach to Claims Reuse

As a discipline, human-computer interaction produces creative and innovative designs that could provide a reusable collection of design knowledge on which future efforts could build. It is unfortunate that so much of this knowledge is not fully reused by designers today. To encourage the use of previously identified HCI knowledge, we propose a model of reuse building on Carroll?s notion of claims, design knowledge components that capture the positive and negative psychological effects of design features. We address four challenges associated with reuse in a library of claims, adopted from software engineering?a discipline in which the notion of reuse has been prevalent for quite some time. Building on Krueger?s definition of reuse and his conceptualization of four key aspects?abstraction, selection, specification, and integration?we propose a reuse approach based on incorporating these four aspects into the design process. To abstract, select, specify and integrate claims, we identify claim relationships, descriptions of connections between claims. We portray how claim relationships can be used to aid in identifying claim types, searching for claims, creating new claims, and aggregating claims. By integrating relationships into a claims library, we demonstrate how they can be applied to assist claims reuse and present studies related to each application of the relationships

Exploratory sequential data analysis of user interaction in contemporary BIM applications

Creation oriented software allows the user to work according to their own vision and rules. From the perspective of software analysis, this is challenging because there is no certainty as to how the users are using the software and what kinds of workflows emerge among different users. The aim of this thesis was to study and identify the potential of sequential event pattern data extraction analysis from expert field creation oriented software in the field of Building Information Modeling (BIM). The thesis additionally introduces a concept evaluation model for detecting repetition based usability disruption. Finally, the work presents an implementation of sequential pattern mining based user behaviour analysis and machine learning predictive application using state of the art algorithms. The thesis introduces a data analysis implementation that is built upon inspections of Sequential or Exploratory Sequential Data Analysis (SDA or ESDA) based theory in usability studies. The study implements a test application specific workflow sequence detection and database transfer approach. The paper uses comparative modern mining algorithms known as BIDE and TKS for sequential pattern discovery. Finally, the thesis utilizes the created sequence database to create user detailing workflow predictions using a CPT+ algorithm. The main contribution of the thesis outcome is to open scalable options for both software usability and product development to automatically recognize and predict usability and workflow related information, deficiencies and repetitive workflow. By doing this, more quantifiable metrics can be revealed in relation to software user interface behavior analytics.Luomiseen perustuva ohjelmisto mahdollistaa käyttäjän työskentelyn oman visionsa ja sääntöjensä mukaisesti. Ohjelmien analysoinnin kannalta tämä on haastavaa, koska ei ole varmuutta siitä, kuinka ohjelmistoa tarkalleen käytetään ja millaisia työskentelytapoja ohjelmiston eri käyttäjäryhmille voi syntyä. Opinnäytetyön tavoitteena oli tutkia ja identifioida toistuvien käyttäjätapahtumasekvenssien analyysipotentiaalia tietomallinnukseen keskittyvässä luomispoh jaisessa ohjelmistossa. Opinnäyte esittelee myös evaluointimallikonseptin, jonka avulla on mahdollista tunnistaa toistuvasta käyttäytymisestä aiheutuvia käytettävyysongelmia. Lopuksi työssä esitellään sekvenssianalyysiin perustuva ohjelmiston käyttäjän toiminta-analyysi sekä ennustava koneoppimisen sovellus. Opinnäytetyössä esitellään data-analyysisovellus, joka perustuu käytettävyystutkimuksessa esiintyvien toistuvien sekvenssien tai kokeellisesti toistuvien sekvenssien analyysiteorian tarkasteluun. Sovelluksen toteutus on tehty eritoten työssä käytetylle ohjelmistolle, jossa käyttäjän detaljointitapahtumista muodostetaan sekvenssejä sekvenssitietokannan luomiseksi. Työssä käytetään sekvenssien toistuvuusanalyysiin moderneja louhintamenetelmiä nimeltään BIDE ja TKS. Lopuksi työssä hyödynnetään luotua sekvenssitietokantaa myös käyttäjän detaljointityön ennustamista varten käyttämällä CPT+ algoritmia. Opinnäytetyön tulosten pohjalta pyritään löytämään vaihtoehtoja käytettävyyden ja tuotekehityksen päätöksenteon tietopohjaiseksi tueksi tunnistamalla ja ennusta malla käyttäjien toimintaa ohjelmistossa. Löydetyn informaation avulla on mahdollista ilmaista käytettävyyteen liittyviä ongelmia kvantitatiivisen tiedon valossa

Quantifying, Modeling and Managing How People Interact with Visualizations on the Web

The growing number of interactive visualizations on the web has made it possible for the general public to access data and insights that were once only available to domain experts. At the same time, this rise has yielded new challenges for visualization creators, who must now understand and engage a growing and diverse audience. To bridge this gap between creators and audiences, we explore and evaluate components of a design-feedback loop that would enable visualization creators to better accommodate their audiences as they explore the visualizations. In this dissertation, we approach this goal by quantifying, modeling and creating tools that manage people’s open-ended explorations of visualizations on the web. In particular, we: 1. Quantify the effects of design alternatives on people’s interaction patterns in visualizations. We define and evaluate two techniques: HindSight (encoding a user’s interaction history) and text-based search, where controlled experiments suggest that design details can significantly modulate the interaction patterns we observe from participants using a given visualization. 2. Develop new metrics that characterize facets of people’s exploration processes. Specifically, we derive expressive metrics describing interaction patterns such as exploration uniqueness, and use Bayesian inference to model distributional effects on interaction behavior. Our results show that these metrics capture novel patterns in people’s interactions with visualizations. 3. Create tools that manage and analyze an audience’s interaction data for a given visualization. We develop a prototype tool, ReVisIt, that visualizes an audience’s interactions with a given visualization. Through an interview study with visualization creators, we found that ReVisIt make creators aware of individual and overall trends in their audiences’ interaction patterns. By establishing some of the core elements of a design-feedback loop for visualization creators, the results in this research may have a tangible impact on the future of publishing interactive visualizations on the web. Equipped with techniques, metrics, and tools that realize an initial feedback loop, creators are better able to understand the behavior and user needs, and thus create visualizations that make data and insights more accessible to the diverse audiences on the web