Using Vision System Technologies for Offset Approaches in Low Visibility Operations

Flight deck-based vision systems, such as Synthetic Vision Systems (SVS) and Enhanced Flight Vision Systems (EFVS), have the potential to provide additional margins of safety for aircrew performance and enable the implementation of operational improvements for low visibility surface, arrival, and departure operations in the terminal environment with equivalent efficiency to visual operations. Twelve air transport-rated crews participated in a motion-base simulation experiment to evaluate the use of SVS/EFVS in Next Generation Air Transportation System low visibility approach and landing operations at Chicago O'Hare airport. Three monochromatic, collimated head-up display (HUD) concepts (conventional HUD, SVS HUD, and EFVS HUD) and three instrument approach types (straight-in, 3-degree offset, 15-degree offset) were experimentally varied to test the efficacy of the SVS/EFVS HUD concepts for offset approach operations. The findings suggest making offset approaches in low visibility conditions with an EFVS HUD or SVS HUD appear feasible. Regardless of offset approach angle or HUD concept being flown, all approaches had comparable ILS tracking during the instrument segment and were within the lateral confines of the runway with acceptable sink rates during the visual segment of the approach. Keywords: Enhanced Flight Vision Systems; Synthetic Vision Systems; Head-up Display; NextGe

Test and Evaluation Metrics of Crew Decision-Making And Aircraft Attitude and Energy State Awareness

NASA has established a technical challenge, under the Aviation Safety Program, Vehicle Systems Safety Technologies project, to improve crew decision-making and response in complex situations. The specific objective of this challenge is to develop data and technologies which may increase a pilot's (crew's) ability to avoid, detect, and recover from adverse events that could otherwise result in accidents/incidents. Within this technical challenge, a cooperative industry-government research program has been established to develop innovative flight deck-based counter-measures that can improve the crew's ability to avoid, detect, mitigate, and recover from unsafe loss-of-aircraft state awareness - specifically, the loss of attitude awareness (i.e., Spatial Disorientation, SD) or the loss-of-energy state awareness (LESA). A critical component of this research is to develop specific and quantifiable metrics which identify decision-making and the decision-making influences during simulation and flight testing. This paper reviews existing metrics and methods for SD testing and criteria for establishing visual dominance. The development of Crew State Monitoring technologies - eye tracking and other psychophysiological - are also discussed as well as emerging new metrics for identifying channelized attention and excessive pilot workload, both of which have been shown to contribute to SD/LESA accidents or incidents

Synthetic and Enhanced Vision Systems for NextGen (SEVS) Simulation and Flight Test Performance Evaluation

The Synthetic and Enhanced Vision Systems for NextGen (SEVS) simulation and flight tests are jointly sponsored by NASA's Aviation Safety Program, Vehicle Systems Safety Technology project and the Federal Aviation Administration (FAA). The flight tests were conducted by a team of Honeywell, Gulfstream Aerospace Corporation and NASA personnel with the goal of obtaining pilot-in-the-loop test data for flight validation, verification, and demonstration of selected SEVS operational and system-level performance capabilities. Nine test flights (38 flight hours) were conducted over the summer and fall of 2011. The evaluations were flown in Gulfstream.s G450 flight test aircraft outfitted with the SEVS technology under very low visibility instrument meteorological conditions. Evaluation pilots flew 108 approaches in low visibility weather conditions (600 ft to 2400 ft visibility) into various airports from Louisiana to Maine. In-situ flight performance and subjective workload and acceptability data were collected in collaboration with ground simulation studies at LaRC.s Research Flight Deck simulator

Evaluation of a Head-Worn Display with Ambient Vision Cues for Unusual Attitude Recovery

A Commercial Aviation Safety Team (CAST) study of 18 loss-of-control events determined that a lack of external visual references was a contributing factor in 17 of these events. CAST recommended that manufacturers should develop and implement virtual day-VMC display systems, such as synthetic vision (SV) or equivalent systems (CAST Safety Enhancement, SE-200). In support of this recommended action, CAST has requested studies to define minimum requirements for virtual day-visual meteorological conditions (VMC) displays to improve flight crew awareness of airplane attitude. NASAs research in Virtual day-VMC displays, known as synthetic vision systems, are intended to support intuitive flight crew attitude awareness similar to a day-VMC-like environment, especially if they could be designed to create visual dominance. A study was conducted to evaluate the utility of ambient vision (AV) cues paired with virtual Head-Up Display (HUD) symbology on a prototype head-worn display (HWD) during recovery from unusual attitudes in a simulated environment. The virtual-HUD component meets the requirement that the HWD may be used as an equivalent display to the HUD. The presence of AV cueing leverages the potential that a HWD has over the HUD for spatial disorientation prevention. The simulation study was conducted as a single-pilot operation, under realistic flight scenarios, with off-nominal events occurring that were capable of inducing unusual attitudes. Independent variables of the experiment included: 1) AV capability (on vs off) 2) AV display opaqueness (transparent vs opaque) and display location (HWD vs traditional head- down displays); AV cues were only present when the HWD was being worn by the subject pilot

Visual Advantage of Enhanced Flight Vision System During NextGen Flight Test Evaluation

Synthetic Vision Systems and Enhanced Flight Vision System (SVS/EFVS) technologies have the potential to provide additional margins of safety for aircrew performance and enable operational improvements for low visibility operations in the terminal area environment. Simulation and flight tests were jointly sponsored by NASA's Aviation Safety Program, Vehicle Systems Safety Technology project and the Federal Aviation Administration (FAA) to evaluate potential safety and operational benefits of SVS/EFVS technologies in low visibility Next Generation Air Transportation System (NextGen) operations. The flight tests were conducted by a team of Honeywell, Gulfstream Aerospace Corporation and NASA personnel with the goal of obtaining pilot-in-the-loop test data for flight validation, verification, and demonstration of selected SVS/EFVS operational and system-level performance capabilities. Nine test flights were flown in Gulfstream's G450 flight test aircraft outfitted with the SVS/EFVS technologies under low visibility instrument meteorological conditions. Evaluation pilots flew 108 approaches in low visibility weather conditions (600 feet to 3600 feet reported visibility) under different obscurants (mist, fog, drizzle fog, frozen fog) and sky cover (broken, overcast). Flight test videos were evaluated at three different altitudes (decision altitude, 100 feet radar altitude, and touchdown) to determine the visual advantage afforded to the pilot using the EFVS/Forward-Looking InfraRed (FLIR) imagery compared to natural vision. Results indicate the EFVS provided a visual advantage of two to three times over that of the out-the-window (OTW) view. The EFVS allowed pilots to view the runway environment, specifically runway lights, before they would be able to OTW with natural vision

Psychophysiological Sensing and State Classification for Attention Management in Commercial Aviation

Attention-related human performance limiting states (AHPLS) can cause pilots to lose airplane state awareness (ASA), and their detection is important to improving commercial aviation safety. The Commercial Aviation Safety Team found that the majority of recent international commercial aviation accidents attributable to loss of control inflight involved flight crew loss of airplane state awareness, and that distraction of various forms was involved in all of them. Research on AHPLS, including channelized attention, diverted attention, startle / surprise, and confirmation bias, has been recommended in a Safety Enhancement (SE) entitled "Training for Attention Management." To accomplish the detection of such cognitive and psychophysiological states, a broad suite of sensors has been implemented to simultaneously measure their physiological markers during high fidelity flight simulation human subject studies. Pilot participants were asked to perform benchmark tasks and experimental flight scenarios designed to induce AHPLS. Pattern classification was employed to distinguish the AHPLS induced by the benchmark tasks. Unimodal classification using pre-processed electroencephalography (EEG) signals as input features to extreme gradient boosting, random forest and deep neural network multiclass classifiers was implemented. Multi-modal classification using galvanic skin response (GSR) in addition to the same EEG signals and using the same types of classifiers produced increased accuracy with respect to the unimodal case (90 percent vs. 86 percent), although only via the deep neural network classifier. These initial results are a first step toward the goal of demonstrating simultaneous real time classification of multiple states using multiple sensing modalities in high-fidelity flight simulators. This detection is intended to support and inform training methods under development to mitigate the loss of ASA and thus reduce accidents and incidents

Quantifying Pilot Visual Attention in Low Visibility Terminal Operations

Quantifying pilot visual behavior allows researchers to determine not only where a pilot is looking and when, but holds implications for specific behavioral tracking when these data are coupled with flight technical performance. Remote eye tracking systems have been integrated into simulators at NASA Langley with effectively no impact on the pilot environment. This paper discusses the installation and use of a remote eye tracking system. The data collection techniques from a complex human-in-the-loop (HITL) research experiment are discussed; especially, the data reduction algorithms and logic to transform raw eye tracking data into quantified visual behavior metrics, and analysis methods to interpret visual behavior. The findings suggest superior performance for Head-Up Display (HUD) and improved attentional behavior for Head-Down Display (HDD) implementations of Synthetic Vision System (SVS) technologies for low visibility terminal area operations. Keywords: eye tracking, flight deck, NextGen, human machine interface, aviatio

Transition of Attention in Terminal Area NextGen Operations Using Synthetic Vision Systems

This experiment investigates the capability of Synthetic Vision Systems (SVS) to provide significant situation awareness in terminal area operations, specifically in low visibility conditions. The use of a Head-Up Display (HUD) and Head-Down Displays (HDD) with SVS is contrasted to baseline standard head down displays in terms of induced workload and pilot behavior in 1400 RVR visibility levels. Variances across performance and pilot behavior were reviewed for acceptability when using HUD or HDD with SVS under reduced minimums to acquire the necessary visual components to continue to land. The data suggest superior performance for HUD implementations. Improved attentional behavior is also suggested for HDD implementations of SVS for low-visibility approach and landing operations

Design Considerations for Attitude State Awareness and Prevention of Entry into Unusual Attitudes

Loss of control - inflight (LOC-I) has historically represented the largest category of commercial aviation fatal accidents. A review of the worldwide transport airplane accidents (2001-2010) evinced that loss of attitude or energy state awareness was responsible for a large majority of the LOC-I events. A Commercial Aviation Safety Team (CAST) study of 18 worldwide loss-of-control accidents and incidents determined that flight crew loss of attitude awareness or energy state awareness due to lack of external visual reference cues was a significant causal factor in 17 of the 18 reviewed flights. CAST recommended that "Virtual Day-Visual Meteorological Condition" (Virtual Day-VMC) displays be developed to provide the visual cues necessary to prevent loss-of-control resulting from flight crew spatial disorientation and loss of energy state awareness. Synthetic vision or equivalent systems (SVS) were identified for a design "safety enhancement" (SE-200). Part of this SE involves the conduct of research for developing minimum aviation system performance standards (MASPS) for these flight deck display technologies to aid flight crew attitude and energy state awareness similar to that of a virtual day-VMC-like environment. This paper will describe a novel experimental approach to evaluating a flight crew's ability to maintain attitude awareness and to prevent entry into unusual attitudes across several SVS optical flow design considerations. Flight crews were subjected to compound-event scenarios designed to elicit channelized attention and startle/surprise within the crew. These high-fidelity scenarios, designed from real-world events, enable evaluation of the efficacy of SVS at improving flight crew attitude awareness to reduce the occurrence of LOC-I incidents in commercial flight operations

Flight Crew Workload, Acceptability, and Performance When Using Data Comm in a High-Density Terminal Area Simulation

This document describes a collaborative FAA/NASA experiment using 22 commercial airline pilots to determine the effect of using Data Comm to issue messages during busy, terminal area operations. Four conditions were defined that span current day to future flight deck equipage: Voice communication only, Data Comm only, Data Comm with Moving Map Display, and Data Comm with Moving Map displaying taxi route. Each condition was used in an arrival and a departure scenario at Boston Logan Airport. Of particular interest was the flight crew response to D-TAXI, the use of Data Comm by Air Traffic Control (ATC) to send taxi instructions. Quantitative data was collected on subject reaction time, flight technical error, operational errors, and eye tracking information. Questionnaires collected subjective feedback on workload, situation awareness, and acceptability to the flight crew for using Data Comm in a busy terminal area. Results showed that 95% of the Data Comm messages were responded to by the flight crew within one minute and 97% of the messages within two minutes. However, post experiment debrief comments revealed almost unanimous consensus that two minutes was a reasonable expectation for crew response. Flight crews reported that Expected D-TAXI messages were useful, and employment of these messages acceptable at all altitude bands evaluated during arrival scenarios. Results also indicate that the use of Data Comm for all evaluated message types in the terminal area was acceptable during surface operations, and during arrivals at any altitude above the Final Approach Fix, in terms of response time, workload, situation awareness, and flight technical performance. The flight crew reported the use of Data Comm as implemented in this experiment as unacceptable in two instances: in clearances to cross an active runway, and D-TAXI messages between the Final Approach Fix and 80 knots during landing roll. Critical cockpit tasks and the urgency of out-the window scan made the additional head down time to respond to Data Comm messages undesirable during these events. However, most crews also stated that Data Comm messages without an accompanying audio chime and no expectation of an immediate response could be acceptable even during these events