Validating Human-device Interfaces with Model Checking and Temporal Logic Properties Automatically Generated from Task Analytic Models

ABSTRACT: When evaluating designs of human-device interfaces for safety critical systems, it is very important that they be valid: support the goal-directed tasks they were designed to facilitate. Model checking is a type of formal analysis that is used to mathematically prove whether or not a model of a system does or does not satisfy a set of specification properties, usually written in a temporal logic. In the analysis of human-automation interaction, model checkers have been used to formally verify that human-device interface models are valid with respect to goal-directed tasks encoded in temporal logic properties. All of the previous work in this area has required that analysts manually specify these properties. Given the semantics of temporal logic and the complexity of task analytic behavior models, this can be very difficult. This paper describes a method that allows temporal logic properties to be automatically generated from task analytic models created early in the system design process. This allows analysts to use model checkers to validate that modeled human-device interfaces will allow human operators to successfully perform the necessary tasks with the system. The use of the method is illustrated with a patient controlled analgesia pump programming example. The method is discussed and avenues for future work are described

A formal approach to discovering simultaneous additive masking between auditory medical alarms

publisher: Elsevier articletitle: A formal approach to discovering simultaneous additive masking between auditory medical alarms journaltitle: Applied Ergonomics articlelink: http://dx.doi.org/10.1016/j.apergo.2016.07.008 content_type: article copyright: © 2016 Elsevier Ltd. All rights reserved

Visual Analysis of Hyperproperties for Understanding Model Checking Results

Model checkers provide algorithms for proving that a mathematical model of a system satisfies a given specification. In case of a violation, a counterexample that shows the erroneous behavior is returned. Understanding these counterexamples is challenging, especially for hyperproperty specifications, i.e., specifications that relate multiple executions of a system to each other. We aim to facilitate the visual analysis of such counterexamples through our HYPERVIS tool, which provides interactive visualizations of the given model, specification, and counterexample. Within an iterative and interdisciplinary design process, we developed visualization solutions that can effectively communicate the core aspects of the model checking result. Specifically, we introduce graphical representations of binary values for improving pattern recognition, color encoding for better indicating related aspects, visually enhanced textual descriptions, as well as extensive cross-view highlighting mechanisms. Further, through an underlying causal analysis of the counterexample, we are also able to identify values that contributed to the violation and use this knowledge for both improved encoding and highlighting. Finally, the analyst can modify both the specification of the hyperproperty and the system directly within HYPERVIS and initiate the model checking of the new version. In combination, these features notably support the analyst in understanding the error leading to the counterexample as well as iterating the provided system and specification. We ran multiple case studies with HYPERVIS and tested it with domain experts in qualitative feedback sessions. The participants’ positive feedback confirms the considerable improvement over the manual, text-based status quo and the value of the tool for explaining hyperproperties

Evaluating humanhuman communication protocols with miscommunication generation and model checking

Abstract. Human-human communication is critical to safe operations in domains such as air transportation where airlines develop and train pilots on communication procedures with the goal to ensure that they check that verbal air traffic clearances are correctly heard and executed. Such communication protocols should be designed to be robust to miscommunication. However, they can fail in ways unanticipated by designers. In this work, we present a method for modeling human-human communication protocols using the Enhanced Operator Function Model with Communications (EOFMC), a task analytic modeling formalism that can be interpreted by a model checker. We describe how miscommunications can be generated from instantiated EOFMC models of human-human communication protocols. Using an air transportation example, we show how model checking can be used to evaluate if a given protocol will ensure successful communication. Avenues of future research are explored

Model-Based Usability Analysis of Safety-Critical Systems: A Formal Methods Framework

Complex, safety-critical systems are designed with a broad range of automated and configurable components, and usability problems often emerge for the end user during setup, operation, and troubleshooting procedures. Usability evaluations should consider the entire human-device interface including displays, controls, hardware configurations, and user documentation/procedures. To support the analyst, human factors researchers have developed a set of methods and measures for evaluating human-system interface usability, while formal methods researchers have developed a set of model-based technologies that enable mathematical verification of desired system behaviors. At the intersection of these disciplines, an evolving set of model-based frameworks enable highly automated verification of usability early in the design cycle. Models can be abstracted to enable broad coverage of possible problems, while measures can be formally verified to "prove" that the system is usable. Currently, frameworks cover a subset of the target system and user behaviors that must be modeled to ensure usability: procedures, visual displays, user controls, automation, and possible interactions among them. Similarly, verification methodologies focus on a subset of potential usability problems with respect to modeled interactions. This work provides an integrated formal methods framework enabling the holistic modeling and verification of safety-critical system usability. Building toward the framework, a set of five, novel approaches extend the capabilities of extant frameworks in different ways. Each approach is demonstrated in a medical device case study to show how the methods can be employed to identify potential usability problems in existing systems. A formal approach to documentation navigation models an end user navigating through a printed or electronic document and verifies page reachability. A formal approach to procedures in documentation models an end user executing steps as written and aids in identifying problems involving what device components are identified in task descriptions, what system configurations are addressed, and what temporal orderings of procedural steps could be improved. A formal approach to hardware configurability models end-user motor capabilities, relationships among the user and device components in the spatial environment, and opportunities for the user to physically manipulate components. An encoding tool facilitates the modeling process, while a verification methodology aids in ensuring that configurable hardware supports correct end- user actions and prevents incorrect ones. A formal approach to interface understandability models what information is provided to the end user through visual, audible, and haptic sensory channels, including explanations provided in accompanying documentation. An encoding tools facilitates the development of models and specifications, while the verification methodology aids in ensuring that what is displayed on the device is consistent; and, if needed, an explanation of what is displayed is provided in documentation. A formal approach to controlled actuators leverages an existing modeling technique and data collected from other engineering activities to model actuator dynamics mapping to referent data. An encoding tool facilitates model development, and a verification methodology aids in validating the model with respect to source data. Finally, new methodologies are combined within the integrated framework. A model architecture supports the analyst in representing a broad range of interactions among constituent framework models, and a set of ten specifications is developed to enable holistic usability verification. An implementation of the framework is demonstrated within a case study based on a medical device under development. This application shows how the framework could be utilized early in the design of a safety-critical system, without the need for a fully implemented device or a team of human evaluators.Ph.D., Biomedical Science -- Drexel University, 201

Computer Aided Verification

This open access two-volume set LNCS 13371 and 13372 constitutes the refereed proceedings of the 34rd International Conference on Computer Aided Verification, CAV 2022, which was held in Haifa, Israel, in August 2022. The 40 full papers presented together with 9 tool papers and 2 case studies were carefully reviewed and selected from 209 submissions. The papers were organized in the following topical sections: Part I: Invited papers; formal methods for probabilistic programs; formal methods for neural networks; software Verification and model checking; hyperproperties and security; formal methods for hardware, cyber-physical, and hybrid systems. Part II: Probabilistic techniques; automata and logic; deductive verification and decision procedures; machine learning; synthesis and concurrency. This is an open access book

Proceedings of Monterey Workshop 2001 Engineering Automation for Sofware Intensive System Integration

The 2001 Monterey Workshop on Engineering Automation for Software Intensive System Integration was sponsored by the Office of Naval Research, Air Force Office of Scientific Research, Army Research Office and the Defense Advance Research Projects Agency. It is our pleasure to thank the workshop advisory and sponsors for their vision of a principled engineering solution for software and for their many-year tireless effort in supporting a series of workshops to bring everyone together.This workshop is the 8 in a series of International workshops. The workshop was held in Monterey Beach Hotel, Monterey, California during June 18-22, 2001. The general theme of the workshop has been to present and discuss research works that aims at increasing the practical impact of formal methods for software and systems engineering. The particular focus of this workshop was "Engineering Automation for Software Intensive System Integration". Previous workshops have been focused on issues including, "Real-time & Concurrent Systems", "Software Merging and Slicing", "Software Evolution", "Software Architecture", "Requirements Targeting Software" and "Modeling Software System Structures in a fastly moving scenario".Office of Naval ResearchAir Force Office of Scientific Research Army Research OfficeDefense Advanced Research Projects AgencyApproved for public release, distribution unlimite