Modelling and systematic analysis of interactive systems

Two aspects of our research concern the application of formal methods in human-computer interaction. The first aspect is the modelling and analysis of interactive devices with a particular emphasis on the user device dyad. The second is the modelling and analysis of ubiquitous systems where there are many users, one might say crowds of users.The common thread of both is to articulate and prove properties of interactive systems, to explore interactive behaviour as it influences the user, with a particular emphasis on interaction failure. The goal is to develop systematic techniques that can be packaged in such a way that they can be used effectively by developers. This “whitepaper” will briefly describe the two approaches and their potential value as well as their limitations and development opportunities

A Review of Milestones in the History of GUI Prototyping Tools

International audiencePrototyping is one of the core activities of User-Centered Design (UCD) processes and an integral component of Human-Computer Interaction (HCI) research. Nonetheless, for many years, prototyping was synonym of paper-based mock-ups and only more recently we can say that dedicated tools for supporting prototyping activities really reach the market. In this paper we propose to analyze the evolution of prototyping tools proposed by the academic community to support research activities and prototyping tools that are aimed and improve the development process of commercial user interfaces. Thus, this paper presents a review of past and current graphical user interface prototyping tools, in order to set up the state of the art in this field, observing fundamental milestones of features over time. For that, we have screened publications presented since 1988 in some of the main HCI conferences and 113 commercial tools available on the web. The results enable a brief comparison of characteristics present in both academic and commercial tools, how they have evolved and what are the gaps that can provide insights for future research and development

Exploring Action Dynamics as an Index of Paired-Associate Learning

Much evidence exists supporting a richer interaction between cognition and action than commonly assumed. Such findings demonstrate that short-timescale processes, such as motor execution, may relate in systematic ways to longer-timescale cognitive processes, such as learning. We further substantiate one direction of this interaction: the flow of cognition into action systems. Two experiments explored match-to-sample paired-associate learning, in which participants learned randomized pairs of unfamiliar symbols. During the experiments, their hand movements were continuously tracked using the Nintendo Wiimote. Across learning, participant arm movements are initiated and completed more quickly, exhibit lower fluctuation, and exert more perturbation on the Wiimote during the button press. A second experiment demonstrated that action dynamics index novel learning scenarios, and not simply acclimatization to the Wiimote interface. Results support a graded and systematic covariation between cognition and action, and recommend ways in which this theoretical perspective may contribute to applied learning contexts

Evidence Report, Risk of Inadequate Design of Human and Automation/Robotic Integration

The success of future exploration missions depends, even more than today, on effective integration of humans and technology (automation and robotics). This will not emerge by chance, but by design. Both crew and ground personnel will need to do more demanding tasks in more difficult conditions, amplifying the costs of poor design and the benefits of good design. This report has looked at the importance of good design and the risks from poor design from several perspectives: 1) If the relevant functions needed for a mission are not identified, then designs of technology and its use by humans are unlikely to be effective: critical functions will be missing and irrelevant functions will mislead or drain attention. 2) If functions are not distributed effectively among the (multiple) participating humans and automation/robotic systems, later design choices can do little to repair this: additional unnecessary coordination work may be introduced, workload may be redistributed to create problems, limited human attentional resources may be wasted, and the capabilities of both humans and technology underused. 3) If the design does not promote accurate understanding of the capabilities of the technology, the operators will not use the technology effectively: the system may be switched off in conditions where it would be effective, or used for tasks or in contexts where its effectiveness may be very limited. 4) If an ineffective interaction design is implemented and put into use, a wide range of problems can ensue. Many involve lack of transparency into the system: operators may be unable or find it very difficult to determine a) the current state and changes of state of the automation or robot, b) the current state and changes in state of the system being controlled or acted on, and c) what actions by human or by system had what effects. 5) If the human interfaces for operation and control of robotic agents are not designed to accommodate the unique points of view and operating environments of both the human and the robotic agent, then effective human-robot coordination cannot be achieved

Collocated Collaboration Analytics: Principles and Dilemmas for Mining Multimodal Interaction Data

© 2019, Copyright © 2017 Taylor & Francis Group, LLC. Learning to collaborate effectively requires practice, awareness of group dynamics, and reflection; often it benefits from coaching by an expert facilitator. However, in physical spaces it is not always easy to provide teams with evidence to support collaboration. Emerging technology provides a promising opportunity to make collocated collaboration visible by harnessing data about interactions and then mining and visualizing it. These collocated collaboration analytics can help researchers, designers, and users to understand the complexity of collaboration and to find ways they can support collaboration. This article introduces and motivates a set of principles for mining collocated collaboration data and draws attention to trade-offs that may need to be negotiated en route. We integrate Data Science principles and techniques with the advances in interactive surface devices and sensing technologies. We draw on a 7-year research program that has involved the analysis of six group situations in collocated settings with more than 500 users and a variety of surface technologies, tasks, grouping structures, and domains. The contribution of the article includes the key insights and themes that we have identified and summarized in a set of principles and dilemmas that can inform design of future collocated collaboration analytics innovations

Assessing the Accuracy of Task Time Prediction of an Emerging Human Performance Modeling Software - CogTool

There is a need for a human performance modeling tool which not only has the ability to accurately estimate skilled user task time for any interface design, but can be used by modelers with little or no programming knowledge and at a minimal cost. To fulfill this need, this research investigated the accuracy of task time prediction of a modeling tool – CogTool - on two versions of an interface design used extensively in the petrochemical industry – DeltaV. CogTool uses the KeyStroke Level Model (KLM) to calculate and generate time predictions based on specified operators. The data collected from a previous study (Koffskey, Ikuma, & Harvey, 2013) that investigated how human participants (24 students and 4 operators) performed on these interfaces (in terms of mean speed in seconds) were compared to CogTool’s numeric time estimate. Three tasks (pump I, pump II and cascade system failures) on each interface for both participant groups were tested on both interfaces (improved and poor), on the general hypothesis that CogTool will make task time predictions for each of the modeled tasks, within a certain range of what actual human participants had demonstrated. The 95% confidence interval (CI) tests of the means were used to determine if the predictions fall within the intervals. The estimated task time from CogTool did not fall within the 95% CI in 9 of 12 cases. Of the 3 that were contained in the acceptable interval, two belonged to the experienced operator group for tasks performed on the improved interface, implying that CogTool was better in predicting the operators’ performance than the students’. A control room monitoring task, by its nature, places great demand on an operator’s mental capacity. This also includes the fact that operators work on multiple screens and/or consoles, sometimes requiring them to commit information to memory that they have to revisit a screen to check on some vital information. In this regard, it is suggested that the one user mental operator for “think time” (estimated as 1.2sec), should be revised in CogTool to accommodate the demand on the operator. For this reason, the present CogTool prediction did not meet expectations in estimating control room operator task time, but it however succeeded in showing where the poor interface could be improved by comparing the detailed steps to the improved interface

Human systems integration: training and education needs analysis

Over the past decade, the military (both US and foreign) has developed a wide range of tools, techniques, and technologies for integrating human factors into systems engineering. Human Systems Integration (HSI) came forth as a new multidisciplinary field of study composed of several basic areas including Human Factors Engineering, System Safety, Health Hazards, Personnel Survivability, Manpower, Personnel, Training, and Habitability. An online survey was designed to examine education and training in HSI. 19 HSI experts responded to the survey. The feedback showed that there is a lack of high qualified practitioners in HSI. Many HSI practitioners are lacking of required skills due to limited resources of education and training in HSI area. A common opinion shared by all the experts is that there are limited programs in HSI or the education programs are not focused on HSI. Many major universities do not have an adequate program. Majority of the HSI programs are focused on mostly Human Factors and are not really on HSI. All aspects of the domains in HSI are not covered in the programs. The experts recommend more applications or hands-on. A series of HSI education and training programs are discussed and recommendations are made to provide a path of future improvement for future students and employers in the field of HSI. Predictions are also made concerning the potential that HSI and Human Factors could be merged into one systematic science that could be used in both industrial and military complex systems --Abstract, page iii

Understanding Eye Gaze Patterns in Code Comprehension

Program comprehension is a sub-field of software engineering that seeks to understand how developers understand programs. Comprehension acts as a starting point for many software engineering tasks such as bug fixing, refactoring, and feature creation. The dissertation presents a series of empirical studies to understand how developers comprehend software in realistic settings. The unique aspect of this work is the use of eye tracking equipment to gather fine-grained detailed information of what developers look at in software artifacts while they perform realistic tasks in an environment familiar to them, namely a context including both the Integrated Development Environment (Eclipse or Visual Studio) and a web browser (Google Chrome). The iTrace eye tracking infrastructure is used for certain eye tracking studies on large code files as it is able to handle page scrolling and context switching. The first study is a classroom-based study on how students actively trained in the classroom understand grouped units of C++ code. Results indicate students made many transitions between lines that were closer together, and were attracted the most to if statements and to a lesser extent assignment code. The second study seeks to understand how developers use Stack Overflow page elements to build summaries of open source project code. Results indicate participants focused more heavily on question and answer text, and the embedded code, more than they did the title, question tags, or votes. The third study presents a larger code summarization study using different information contexts: Stack Overflow, bug repositories and source code. Results show participants tended to visit up to two codebase files in either the combined or isolated codebase session, but visit more bug report pages, and spend longer time on new Stack Overflow pages they visited, when given either these two treatments in isolation. In the combined session, time spent on the one or two codebase files they viewed dominated the session time. Information learned from tracking developers\u27 gaze in these studies can form foundations for developer behavior models, which we hope can later inform recommendations for actions one might take to achieve workflow goals in these settings. Advisor: Bonita Shari