Validation of cognitive models for collaborative hybrid systems with discrete human input

We present a method to validate a cognitive model, based on the cognitive architecture ACT-R, in dynamic humanautomation systems with discrete human input. We are inspired by the general problem of K-choice games as a proxy for many decision making applications in dynamical systems. We model the human as a Markovian controller based on gathered experimental data, that is, a non-deterministic control input with known likelihoods of control actions associated with certain configurations of the state-space. We use reachability analysis to predict the outcome of the resulting discrete-time stochastic hybrid system, in which the outcome is defined as a function of the system trajectory. We suggest that the resulting expected outcomes can be used to validate the cognitive model against actual human subject data. We apply our method to a twochoice game in which the human is tasked with maximizing net coverage of a robotic swarm that can operate under rendezvous or deployment dynamics. We validate the corresponding ACTR cognitive model generated with the data from eight human subjects. The novelty of this work is 1) a method to compute expected outcome in a hybrid dynamical system with a Markov chain model of the human's discrete choice, and 2) application of this method to validation of cognitive models with a database of actual human subject data

Abstraction of analytical models from cognitive models of human control of robotic swarms

In order to formally validate cyber-physical systems, analytically tractable models of human control are desirable. While those models can be abstracted directly from human data, limitations on the amount and reliability of data can lead to over-fitting and lack of generalization. We introduce a methodology for deriving formal models of human control of cyberphysical systems based on the use of cognitive models. Analytical models such as Markov models can be derived from an instance-based learning model of the task built using the ACT-R cognitive architecture. The approach is illustrated in the context of a robotic control task involving the choice of two options to control a robotic swarm. The cognitive model and various forms of the analytical model are validated against each other and against human performance data. The current limitations of the approach are discussed as well as its implications for the automated validation of cyber-physical systems

A computational model based on human performance for fluid management in critical care

Computational simulation is one of the most important ways of reproducing the dynamic responses of a Cyber Physical System using a model of the system. The simulation discovers areas of differential system performance and allows linking such performance back to system characteristics. In the medical domain, patient simulators are used to train physicians in patient management. One critical question is how to verify these systems under realistic human (physician) input. This requires the creation of realistic human models that would be able to capture human cognitive and decision abilities and limitations. Verification of such an overall physician-patient model would result in two advantages: (a) since physicians realistically would not give all possible inputs to the system, verification could be more efficient and (b) the verification may uncover areas of poor human performance. In this paper, we describe our methodology and results in creating a computational model of human fluid management in critical care, based on human experiments

Human Performance Models of Pilot Behavior

Five modeling teams from industry and academia were chosen by the NASA Aviation Safety and Security Program to develop human performance models (HPM) of pilots performing taxi operations and runway instrument approaches with and without advanced displays. One representative from each team will serve as a panelist to discuss their team s model architecture, augmentations and advancements to HPMs, and aviation-safety related lessons learned. Panelists will discuss how modeling results are influenced by a model s architecture and structure, the role of the external environment, specific modeling advances and future directions and challenges for human performance modeling in aviation

Cognitive architectures as Lakatosian research programmes: two case studies

Cognitive architectures - task-general theories of the structure and function of the complete cognitive system - are sometimes argued to be more akin to frameworks or belief systems than scientific theories. The argument stems from the apparent non-falsifiability of existing cognitive architectures. Newell was aware of this criticism and argued that architectures should be viewed not as theories subject to Popperian falsification, but rather as Lakatosian research programs based on cumulative growth. Newell's argument is undermined because he failed to demonstrate that the development of Soar, his own candidate architecture, adhered to Lakatosian principles. This paper presents detailed case studies of the development of two cognitive architectures, Soar and ACT-R, from a Lakatosian perspective. It is demonstrated that both are broadly Lakatosian, but that in both cases there have been theoretical progressions that, according to Lakatosian criteria, are pseudo-scientific. Thus, Newell's defense of Soar as a scientific rather than pseudo-scientific theory is not supported in practice. The ACT series of architectures has fewer pseudo-scientific progressions than Soar, but it too is vulnerable to accusations of pseudo-science. From this analysis, it is argued that successive versions of theories of the human cognitive architecture must explicitly address five questions to maintain scientific credibility