Integrating planning, execution, and learning

To achieve the goal of building an autonomous agent, the usually disjoint capabilities of planning, execution, and learning must be used together. An architecture, called MAX, within which cognitive capabilities can be purposefully and intelligently integrated is described. The architecture supports the codification of capabilities as explicit knowledge that can be reasoned about. In addition, specific problem solving, learning, and integration knowledge is developed

A time series feature of variability to detect two types of boredom from motion capture of the head and shoulders

Boredom and disengagement metrics are crucial to the correctly timed implementation of adaptive interventions in interactive systems. psychological research suggests that boredom (which other HCI teams have been able to partially quantify with pressure-sensing chair mats) is actually a composite: lethargy and restlessness. Here we present an innovative approach to the measurement and recognition of these two kinds of boredom, based on motion capture and video analysis of changes in head and shoulder positions. Discrete, three-minute, computer-presented stimuli (games, quizzes, films and music) covering a spectrum from engaging to boring/disengaging were used to elicit changes in cognitive/emotional states in seated, healthy volunteers. Interaction with the stimuli occurred with a handheld trackball instead of a mouse, so movements were assumed to be non-instrumental. Our results include a feature (standard deviation of windowed ranges) that may be more specific to boredom than mean speed of head movement, and that could be implemented in computer vision algorithms for disengagement detection

Noninvasive Physiological Measures And Workload Transitions:an Investigation Of Thresholds Using Multiple Synchronized Sensors

The purpose of this study is to determine under what conditions multiple minimally intrusive physiological sensors can be used together and validly applied for use in areas which rely on adaptive systems including adaptive automation and augmented cognition. Specifically, this dissertation investigated the physiological transitions of operator state caused by changes in the level of taskload. Three questions were evaluated including (1) Do differences exist between physiological indicators when examined between levels of difficulty? (2) Are differences of physiological indicators (which may exist) between difficulty levels affected by spatial ability? (3) Which physiological indicators (if any) account for variation in performance on a spatial task with varying difficulty levels? The Modular Cognitive State Gauge model was presented and used to determine which basic physiological sensors (EEG, ECG, EDR and eye-tracking) could validly assess changes in the utilization of two-dimensional spatial resources required to perform a spatial ability dependent task. Thirty-six volunteers (20 female, 16 male) wore minimally invasive physiological sensing devices while executing a challenging computer based puzzle task. Specifically, participants were tested with two measures of spatial ability, received training, a practice session, an experimental trial and completed a subjective workload survey. The results of this experiment confirmed that participants with low spatial ability reported higher subjective workload and performed poorer when compared to those with high spatial ability. Additionally, there were significant changes for a majority of the physiological indicators between two difficulty levels and most importantly three measures (EEG, ECG and eye-tracking) were shown to account for variability in performance on the spatial task

Reducing Human/Pilot Errors in Aviation Using Augmented Cognition and Automation Systems in Aircraft Cockpit

Human errors cause the majority of aviation accidents. Augmented cognition and automation systems enhance pilot performance by evaluating system limitations and flight precision and performance. This study examines the human-machine interface in cockpit design using the tenets of augmented cognition and automation systems theory in terms of task allocation, attentional resources, and situational awareness. The study compares how these principles apply to and interact with each other and with a human/pilot in a closed-loop system. We present a method for integrating augmented cognition systems into airplane flight management systems. We demonstrate systems enhancement with an experiment in which test pilots flew two simulated flights, once without and once with an augmented cognition system. We measured pilot and airplane performance, pilots’ situational awareness, workload management, pilots’ use of cockpit checklists, and flight precision along four axes: (1) altitude, (2) course, (3) radial/bearing and heading, and (4) airspeed

Predicting Cognitive Workload with Measures from Functional Near-Infrared Spectroscopy (fNIRS) and Heart Rate

The objective of this study was to assess low to high levels of Cognitive Workload by measuring heart rate and cortical blood flow in real-time. Four conditions were implemented into a within-subjects experimental design. Two conditions of difficulty and two conditions of trial order were used to illicit different levels of workload which will be analyzed with psychophysiological equipment. Functional Near-Infrared Spectroscopy (fNIRS) has become more prominent for measuring the blood oxygenation levels in the prefrontal cortex of individuals operating in hazardous work environments, students with learning disabilities, and in research for military training. This is due to the fNIR device being highly mobile, inexpensive, and able to produce a high-spatial resolution of the dorsolateral prefrontal cortex during executive functioning. Heart Rate will be measured by an Electrocardiogram, which will be used in concordance with fNIR oxygenation levels to predict if an individual is in a condition that produces low or high mental workload. Successfully utilizing heart rate and blood oxygenation data as predictors of cognitive workload may validate implementing multiple physiological devices together in real-time and may be a more accurate solution for preventing excessive workload

Development of Requirements to Incorporate Neurophysiological Measures in Human Computer Interface Design

This project specifies requirements for testing platforms and facilities that will enable the use of neurophysiological data to help improve human computer-interfaces. The data used to generate these requirements was collected as part of an advanced human factors effort aimed at improving the usability of future releases of the Tactical Tomahawk Weapons Control System (TTWCS). Cognitive state was measured using electrocardiography (EKG), galvanic skin response (GSR), and electroencephalography (EEG), in addition to traditional measures using various subjective and psychological analyses. This project demonstrated the value of neurophysiological measures into the Human Computer Interaction (HCI) design process, including increased objectivity of measures and consistency between measures. Simultaneous neurophysiological and psychological measurements will enable researchers to better understand true usability of an interface and the requirements documented herein will enable such research

Combining computer game-based behavioural experiments with high-density EEG and infrared gaze tracking

Rigorous, quantitative examination of therapeutic techniques anecdotally reported to have been successful in people with autism who lack communicative speech will help guide basic science toward a more complete characterisation of the cognitive profile in this underserved subpopulation, and show the extent to which theories and results developed with the high-functioning subpopulation may apply. This study examines a novel therapy, the "Rapid Prompting Method" (RPM). RPM is a parent-developed communicative and educational therapy for persons with autism who do not speak or who have difficulty using speech communicatively.The technique aims to develop a means of interactive learning by pointing amongst multiple-choice options presented at different locations in space, with the aid of sensory "prompts" which evoke a response without cueing any specific response option. The prompts are meant to draw and to maintain attention to the communicative task–making the communicative and educational content coincident with the most physically salient, attention-capturing stimulus – and to extinguish the sensory–motor preoccupations with which the prompts compete.ideo-recorded RPM sessions with nine autistic children ages 8–14years who lacked functional communicative speech were coded for behaviours of interest

Optimizing The Design Of Multimodal User Interfaces

Due to a current lack of principle-driven multimodal user interface design guidelines, designers may encounter difficulties when choosing the most appropriate display modality for given users or specific tasks (e.g., verbal versus spatial tasks). The development of multimodal display guidelines from both a user and task domain perspective is thus critical to the achievement of successful human-system interaction. Specifically, there is a need to determine how to design task information presentation (e.g., via which modalities) to capitalize on an individual operator\u27s information processing capabilities and the inherent efficiencies associated with redundant sensory information, thereby alleviating information overload. The present effort addresses this issue by proposing a theoretical framework (Architecture for Multi-Modal Optimization, AMMO) from which multimodal display design guidelines and adaptive automation strategies may be derived. The foundation of the proposed framework is based on extending, at a functional working memory (WM) level, existing information processing theories and models with the latest findings in cognitive psychology, neuroscience, and other allied sciences. The utility of AMMO lies in its ability to provide designers with strategies for directing system design, as well as dynamic adaptation strategies (i.e., multimodal mitigation strategies) in support of real-time operations. In an effort to validate specific components of AMMO, a subset of AMMO-derived multimodal design guidelines was evaluated with a simulated weapons control system multitasking environment. The results of this study demonstrated significant performance improvements in user response time and accuracy when multimodal display cues were used (i.e., auditory and tactile, individually and in combination) to augment the visual display of information, thereby distributing human information processing resources across multiple sensory and WM resources. These results provide initial empirical support for validation of the overall AMMO model and a sub-set of the principle-driven multimodal design guidelines derived from it. The empirically-validated multimodal design guidelines may be applicable to a wide range of information-intensive computer-based multitasking environments

Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks

Biological plastic neural networks are systems of extraordinary computational capabilities shaped by evolution, development, and lifetime learning. The interplay of these elements leads to the emergence of adaptive behavior and intelligence. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed plastic neural networks with a large variety of dynamics, architectures, and plasticity rules: these artificial systems are composed of inputs, outputs, and plastic components that change in response to experiences in an environment. These systems may autonomously discover novel adaptive algorithms, and lead to hypotheses on the emergence of biological adaptation. EPANNs have seen considerable progress over the last two decades. Current scientific and technological advances in artificial neural networks are now setting the conditions for radically new approaches and results. In particular, the limitations of hand-designed networks could be overcome by more flexible and innovative solutions. This paper brings together a variety of inspiring ideas that define the field of EPANNs. The main methods and results are reviewed. Finally, new opportunities and developments are presented