Toward a user-centered design of a weather forecasting decision-support tool

Hazard Services is a software toolkit that integrates information management, hazard alerting, and communication functions into a single user interface. When complete, National Weather Service forecasters across the United States will use Hazard Services for operational issuance of weather and hydrologic alerts, making the system an instrumental part of the threat management process. As a new decision-support tool, incorporating an understanding of user requirements and behavior is an important part of building a system that is usable, allowing users to perform work-related tasks efficiently and effectively. This paper discusses the Hazard Services system and findings from a usability evaluation with a sample of end users. Usability evaluations are frequently used to support software and website development and can provide feedback on a system’s efficiency of use, effectiveness, and learnability. In the present study, a user-testing evaluation assessed task performance in terms of error rates, error types, response time, and subjective feedback from a questionnaire. A series of design recommendations was developed based on the evaluation’s findings. The recommendations not only further the design of Hazard Services, but they may also inform the designs of other decision-support tools used in weather and hydrologic forecasting. Incorporating usability evaluation into the iterative design of decision-support tools, such as Hazard Services, can improve system efficiency, effectiveness, and user experience

Physiological Indicators of Task Demand, Fatigue, and Cognition in Future Digital Manufacturing Environments

As Digital Manufacturing transforms traditionally physical work into more system-monitoring tasks, new methods are required for understanding people's mental workload and prolonged capacity for focused attention. Many physiological measures have shown promise for detecting changes in cognitive state, and recent advances in sensor technology offer minimally-invasive ways to monitor our cognitive activity. Previous research in functional near-infrared spectroscopy, for example, has observed changes in cerebral hemodynamic response during periods of high demand within tasks. This work investigated the relationships among task demand, fatigue, and attention degradation in a sustained attention task, and their effect on heart rate, breathing rate, nose temperature and hemodynamic response in the prefrontal cortex and middle temporal gyrus. Analysis revealed a small but significant effect of fatigue on heart rate relative to baseline, breathing rate and hemodynamic response. Task demand had a small but significant effect on breathing rate and nose temperature, both relative to baseline, but no difference between levels of demand was observed in heart rate or hemodynamic response. Our results provide insight into what physiological data can tell us about cognitive state, ability to focus, and the impact of fatigue over time

Effects of display design on signal detection in flash flood forecasting

The Flooded Locations and Simulated Hydrographs (FLASH) project is a suite of tools that use weather radar-based rainfall estimates to force hydrologic models to predict flash floods in real-time. However, early evaluation of FLASH tools in a series of simulated forecasting operations, it was believed that the data aggregation and visualization methods might have contributed to forecasting a large number of false alarms. The present study addresses the question of how two alternative data aggregation and visualization methods affect signal detection of flash floods. A sample of 30 participants viewed a series of stimuli created from FLASH images and were asked to judge whether or not they predicted significant or insignificant amounts of flash flooding. Analyses revealed that choice of aggregation method did affect probability of detection. Additional visual indicators such as geographic scale of the stimuli and threat level affected the odds of interpreting the model predictions correctly as well as congruence in responses between national and local scale model outputs

Investigating the relationship between eye movements and situation awareness in weather forecasting

Physiological indicators, including eye tracking measures, may provide insight into human decision making and cognition in many domains, including weather forecasting. Situation awareness (SA), a critical component of forecast decision making, is commonly conceptualized as the degree to which information is perceived, understood, and projected into a future context. Drawing upon recent applications of eye tracking in the study of forecaster decision making, we investigate the relationship among eye movement measures, automation, and SA assessed through a freeze probe assessment method. In addition, we explore the relationship between an automated forecasting decision aid use and information seeking behavior.In this study, a sample of professional weather forecasters completed a series of tasks, informed by a set of forecasting decision aids, and with variable access to an experimental automated tool, while an eye tracking system captured data related to eye movements and information usage. At the end of each forecasting task, participants responded to a set of questions related to the environmental situation in the framework of a survey-based assessment technique in order to assess their level of situation awareness. Regression analysis revealed a moderate relationship between the SA measure and eye tracking metrics, supporting the hypothesis that eye tracking may have utility in assessing SA. The results support the use of eye tracking in the assessment of specific and measurable attributes of the decision-making process in weather forecasting. The findings are discussed in light of potential benefits that eye tracking could bring to human performance assessment as well as decision-making research in the forecasting domain

The future of manufacturing: Utopia or dystopia?

Digital Manufacturing Technologies (DMTs) have the potential to transform industry productivity, but their introduction into the workplace is often a complex process, requiring not only technical expertise but also an awareness of ethical and societal challenges surrounding human-system integration. Concerns about the introduction of new technology have been prevalent throughout history, and exploring public perceptions of these technologies can provide insight to help address such cultural anxieties. However, evaluating user perceptions of futuristic technology is difficult, requiring novel approaches to provide context and understanding. In order to explore users’ perceptions of future DMTs, we applied the ContraVision technique in a questionnaire-based study. Participants viewed films, representing fictionalised Utopic and Dystopic visions of what the future of these DMTs might involve, and a questionnaire probed the perceptions of the technologies afterwards. Findings showed that irrespective of the way technology was portrayed, participants had concerns about the ethical and responsible implementation of these tools. Participant responses were analysed to identify key challenges for policy surrounding DMT implementation in the future of manufacturing

The FLASH project: improving the tools for flash flood monitoring and prediction across the United States

This study introduces the Flooded Locations and Simulated Hydrographs (FLASH) project. FLASH is the first system to generate a suite of hydrometeorological products at flash flood scale in real-time across the conterminous United States, including rainfall average recurrence intervals, ratios of rainfall to flash flood guidance, and distributed hydrologic model–based discharge forecasts. The key aspects of the system are 1) precipitation forcing from the National Severe Storms Laboratory (NSSL)’s Multi-Radar Multi-Sensor (MRMS) system, 2) a computationally efficient distributed hydrologic modeling framework with sufficient representation of physical processes for flood prediction, 3) capability to provide forecasts at all grid points covered by radars without the requirement of model calibration, and 4) an open-access development platform, product display, and verification system for testing new ideas in a real-time demonstration environment and for fostering collaborations. This study assesses the FLASH system’s ability to accurately simulate unit peak discharges over a 7-yr period in 1,643 unregulated gauged basins. The evaluation indicates that FLASH’s unit peak discharges had a linear and rank correlation of 0.64 and 0.79, respectively, and that the timing of the peak discharges has errors less than 2 h. The critical success index with FLASH was 0.38 for flood events that exceeded action stage. FLASH performance is demonstrated and evaluated for case studies, including the 2013 deadly flash flood case in Oklahoma City, Oklahoma, and the 2015 event in Houston, Texas—both of which occurred on Memorial Day weekends

Human performance and strategies while solving an aircraft routing and sequencing problem: an experimental approach

As airport resources are stretched to meet increasing demand for services, effective use of ground infrastructure is increasingly critical for ensuring operational efficiency. Work in operations research has produced algorithms providing airport tower controllers with guidance on optimal timings and sequences for flight arrivals, departures, and ground movement. While such decision support systems have the potential to improve operational efficiency, they may also affect users’ mental workload, situation awareness, and task performance. This work sought to identify performance outcomes and strategies employed by human decision makers during an experimental airport ground movement control task with the goal of identifying opportunities for enhancing user-centered tower control decision support systems. To address this challenge, thirty novice participants solved a set of vehicle routing problems presented in the format of a game representing the airport ground movement task practiced by runway controllers. The games varied across two independent variables, network map layout (representing task complexity) and gameplay objective (representing task flexibility), and verbal protocol, visual protocol, task performance, workload, and task duration were collected as dependent variables. A logistic regression analysis revealed that gameplay objective and task duration significantly affected the likelihood of a participant identifying the optimal solution to a game, with the likelihood of an optimal solution increasing with longer task duration and in the less flexible objective condition. In addition, workload appeared unaffected by either independent variable, but verbal protocols and visual observations indicated that high-performing participants demonstrated a greater degree of planning and situation awareness. Through identifying human behavior during optimization problem solving, the work of tower control can be better understood, which, in turn, provides insights for developing decision support systems for ground movement management