Learning structured categories from interrelated features

Issued as Final report, Project G-42-61

The Roles of Motion and Moving Parts in Noun and Verb Meanings

This study contrasts the learning of two different kinds of motion. The first of these we call extrinsic motion, or the motion of one object with respect to another, reference object. The second we call intrinsic motion, or the motion of an object or its parts expressed with respect to the object itself. An experiment tests for people's abilities to associate these two types of motion with nouns and verbs. Subjects were presented with animated events on a computer screen accompanied by sentences involving nouns and verbs. In the learning phase, each noun and verb was related to both an extrinsic motion attribute and an intrinsic motion attribute. Subjects were then tested by presenting them with pairs of events varying on only one of these attributes and asking them which event better exemplified the meaning of a particular noun or verb. The results of this experiment demonstrate a bias to associate verbs with extrinsic motion and to associate nouns with intrinsic motion. These results suggest a division of labor between noun and verb meanings, with verb meanings specialized to encode relational information, while noun meanings are specialized to encode information about objects in isolation

Best Practices for Evaluating Flight Deck Interfaces for Transport Category Aircraft with Particular Relevance to Issues of Attention, Awareness, and Understanding CAST SE-210 Output 2 Report 6 of 6

Attention, awareness, and understanding of the flight crew are a critical contributor to safety and the flight deck plays a critical role in supporting these cognitive functions. Changes to the flight deck need to be evaluated for whether the changed device provides adequate support for these functions. This report describes a set of diverse evaluation methods. The report recommends designing the interface-evaluation to span the phases of the device development, from early to late, and it provides methods appropriate at each phase. It describes the various ways in which an interface or interface component can fail to support awareness as potential issues to be assessed in evaluation. It summarizes appropriate methods to evaluate different issues concerning inadequate support for these functions, throughout the phases of development

Monitoring for Flight Path Management: Inputs to Pilot Monitoring Training

The commercial aviation industry world-wide has identified a need for improved pilot monitoring (e.g., ICAO, 2016). More specifically, aviation safety data indicate that failures in pilots' monitoring for flight path management (FPM) have contributed to a range of undesired outcomes: accidents, major upsets, and non-compliance with ATC guidance. The FAA has further stated that these types of FPM failures are likely to worsen with the increasingly complex air traffic control systems and FPM concepts proposed for NextGen (https://www.faa.gov/nextgen/what_is_nextgen/) operations (e.g., see Hah et al., 2017). An important element of this additional complexity will be the introduction of new automation or artificial intelligence that is intended to work with the flight crew but can add additional monitoring burdens. One potential mitigation for this situation is to enhance pilot training for effective monitoring. NASA Ames Research Center was asked to identify and evaluate training approaches that have the potential to enhance pilots' ability to effectively monitor for FPM (with the result of improved awareness). The focus of this work is to identify, develop or validate training guidance to improve pilot monitoring/awareness regarding FPM and mitigate the recent trend of accidents and incidents, especially Loss of Control (LOC) events. The result of this work should be guidance for FAA training and standards organizationssuch as the Air Carrier Training Aviation Rulemaking Committee (ACT ARC)to aid in reducing the risk of incidents and accidents due to inadequate pilot monitoring/awareness

How Do Different Knowledge Frameworks Help Us Learn From Aviation Line Observations?

Human performance includes actions that increase safety, as well as actions that can reduce safety. Ensuring safety in complex dynamic operations like commercial aviation depends on the ability to institute appropriate responses based on what is learned from flightcrew performance and the contexts in which it occurs. To do this systemically at the organization level requires collecting data on flightcrew performance, developing effective approaches to analyzing those data, and understanding how to translate what has been learned into policies, procedures, and practice. Systematic observation of front-line operators is a vital source of human performance data. Much has been learned from such observations, including methodological principles. Most observations have been based on a framework focused on managing safety challenges and the ensuing unsafe events. A complementary perspective focuses on flexibility and actions that promote continued safe and effective operation. We consider lessons learned about observational methods from an established framework focused on undesired actions and how these might be extended for a framework focused on desired actions

A Model of Monitoring Complex Automated Systems and an Associated Training Approach

Problem. Analysis of recent accidents and incidents have pointed to monitoring failure as a contributing factor. Challenges in monitoring are likely to increase as the complexity of automated systems increases, for airplanes, for UAVs, and other supervisory tasks. Improving monitoring skills though training is one component of reducing monitoring failure and this prompted our research on monitoring and monitoring training. Method. In this research we develop a cognitive characterization of monitoring drawing on detailed discussion with pilots and on research literature. We review research on monitoring and particularly on training monitoring. Based on our characterization of monitoring, we recommend promising approaches to what should be trained and how to train this

Assessment of Alternative Interfaces for Manual Commanding of Spacecraft Systems: Compatibility with Flexible Allocation Policies

Astronauts will be responsible for executing a much larger body of procedures as human exploration moves further from Earth and Mission Control. Efficient, reliable methods for executing these procedures, including manual, automated, and mixed execution will be important. Our interface integrates step-by-step instruction with the means for execution. The research reported here compared manual execution using the new system to a system analogous to the manual-only system currently in use on the International Space Station, to assess whether user performance in manual operations would be as good or better with the new than with the legacy system. The system used also allows flexible automated execution. The system and our data lay the foundation for integrating automated execution into the flow of procedures designed for humans. In our formative study, we found speed and accuracy of manual procedure execution was better using the new, integrated interface over the legacy design

Human-Automation Integration: Principle and Method for Design and Evaluation

Future space missions will increasingly depend on integration of complex engineered systems with their human operators. It is important to ensure that the systems that are designed and developed do a good job of supporting the needs of the work domain. Our research investigates methods for needs analysis. We included analysis of work products (plans for regulation of the space station) as well as work processes (tasks using current software), in a case study of Attitude Determination and Control Officers (ADCO) planning work. This allows comparing how well different designs match the structure of the work to be supported. Redesigned planning software that better matches the structure of work was developed and experimentally assessed. The new prototype enabled substantially faster and more accurate performance in plan revision tasks. This success suggests the approach to needs assessment and use in design and evaluation is promising, and merits investigatation in future research

Evidence Report, Risk of Inadequate Design of Human and Automation/Robotic Integration

The success of future exploration missions depends, even more than today, on effective integration of humans and technology (automation and robotics). This will not emerge by chance, but by design. Both crew and ground personnel will need to do more demanding tasks in more difficult conditions, amplifying the costs of poor design and the benefits of good design. This report has looked at the importance of good design and the risks from poor design from several perspectives: 1) If the relevant functions needed for a mission are not identified, then designs of technology and its use by humans are unlikely to be effective: critical functions will be missing and irrelevant functions will mislead or drain attention. 2) If functions are not distributed effectively among the (multiple) participating humans and automation/robotic systems, later design choices can do little to repair this: additional unnecessary coordination work may be introduced, workload may be redistributed to create problems, limited human attentional resources may be wasted, and the capabilities of both humans and technology underused. 3) If the design does not promote accurate understanding of the capabilities of the technology, the operators will not use the technology effectively: the system may be switched off in conditions where it would be effective, or used for tasks or in contexts where its effectiveness may be very limited. 4) If an ineffective interaction design is implemented and put into use, a wide range of problems can ensue. Many involve lack of transparency into the system: operators may be unable or find it very difficult to determine a) the current state and changes of state of the automation or robot, b) the current state and changes in state of the system being controlled or acted on, and c) what actions by human or by system had what effects. 5) If the human interfaces for operation and control of robotic agents are not designed to accommodate the unique points of view and operating environments of both the human and the robotic agent, then effective human-robot coordination cannot be achieved