A system for synthetic vision and augmented reality in future flight decks

Rockwell Science Center is investigating novel human-computer interaction techniques for enhancing the situational awareness in future flight decks. One aspect is to provide intuitive displays that provide the vital information and the spatial awareness by augmenting the real world with an overlay of relevant information registered to the real world. Such Augmented Reality (AR) techniques can be employed during bad weather scenarios to permit flying in Visual Flight Rules (VFR) in conditions which would normally require Instrumental Flight Rules (IFR). These systems could easily be implemented on heads-up displays (HUD). The advantage of AR systems vs. purely synthetic vision (SV) systems is that the pilot can relate the information overlay to real objects in the world, whereas SV systems provide a constant virtual view, where inconsistencies can hardly be detected. The development of components for such a system led to a demonstrator implemented on a PC. A camera grabs video images which are overlaid with registered information. Orientation of the camera is obtained from an inclinometer and a magnetometer; position is acquired from GPS. In a possible implementation in an airplane, the on-board attitude information can be used for obtaining correct registration. If visibility is sufficient, computer vision modules can be used to fine-tune the registration by matching visual cues with database features. This technology would be especially useful for landing approaches. The current demonstrator provides a frame-rate of 15 fps, using a live video feed as background with an overlay of avionics symbology in the foreground. In addition, terrain rendering from a 1 arc sec. digital elevation model database can be overlaid to provide synthetic vision in case of limited visibility. For true outdoor testing (on ground level), the system has been implemented on a wearable computer

A Flight Test Study to Assess the Utility of an Aircraft Referenced 3D Audio Display to Improve Pilot Performance under High Workload Conditions

A study to assess the utility of an aircraft referenced 3D audio display was undertaken to determine if there could be any improvements to pilot performance when operating under high workload conditions. Test subjects flew a general aviation light twin-engine aircraft under simulated single-pilot instrument flight rule conditions. Workload was elevated by ensuring each test subject had to execute an unexpected missed approach procedure and simultaneously handle a simulated engine failure. Subjective data was gathered using the NASA Task Load Index and a post-flight questionnaire on perceived performance, workload and situational awareness. Objective data on pilot performance was gathered using the research aircraft’s onboard instrumentation system. Within the limitations of having a low number (5) of test subjects available, subjective data results showed a perceived increase in situational awareness, performance, and a statistically significant reduction in workload. Although not statistically significant, the only objective impact to performance was a slight increase in heading control and course intercept. There was no corresponding performance increase in airspeed control, angle of bank control, or improvements to aircraft track. Overall, the results indicate that a 3D audio display would have utility and pilot acceptance as a supplemental navigational display, but would not result in any substantial improvements to pilot performance

A Limited Evaluation of Pilot Technical Performance, Situational Awareness, and Workload when Flying with an Airframe-Referenced 3D Audio Display

The visual workload of general aviation pilots operating alone in adverse conditions is often high. Airframe-referenced 3D audio systems may improve safety and performance by transferring a portion of this workload to the audio modality. These systems use the aircraft’s axes as a reference to present audio directional cues to the pilot via a headset. This experiment compared the technical performance, workload, and situational awareness of pilots as they performed instrument flight procedures with and without an airframe-referenced 3D audio system. Five pilot participants flew six tasks of each audio condition in a Piper Navajo aircraft modified with a 3D audio system. With the 3D audio system, the pilots showed slightly better technical performance and lower workload, as well as improved situational awareness. However, the improvements from 3D audio were not significant when expanded to the entire pilot population. The lack of statistical significance appeared to be the results of a small pilot sample and better than expected pilot performance in both audio conditions. The most important results of the experiment were the modes of the responses to a situational awareness questionnaire consisting of 26 Likert-type questions. Pilots clearly indicated they preferred flying with the 3D system to flying without it, suggesting the airframe-referenced 3D audio system may have been beneficial in completing flight tasks. Further research should be performed to document the system’s impact on pilot performance, especially at higher levels of workload than were evaluated in this study

PC-based aviation training devices (PCATDs): research, development and certification

This paper examines the development of two PCATD’s (one helicopter, one fixed-wing) and their eventual certification by CAA. Certification has demonstrated the potential these devices have for aviation training in New Zealand. Traditionally FTD‘s and PCATD’s have been sourced from foreign companies, and they represent a considerable financial investment for large flying training organisations. The procurement of these simulator types is generally beyond the financial resources of most small to medium sized flying schools. Aviation training in NZ is facing significant financial constraints as well as an increasing demand to simulate complex glass cockpit systems that are now installed in most new General Aviation (GA) aircraft. The development, utilisation and certification of this type of PCATD technology could solve these difficult challenges

AUGMENTING HEADS-UP DISPLAYS WITH INTELLIGENT AGENTS: A HUMAN FACTORS APPROACH

Situational awareness, both tactical and strategic, is essential for humans engaged in complex tasks in civilian and military theaters of operation. Previous work has shown that heads-up displays are effective tools for providing critical information to operators in such situations. Hitherto, heads-up displays have been designed to relay instrument and sensor information to the operator in a topical, timely, and accurate manner. There is a large body of complementary work in the area of human factors that deals with presenting information to a user without detracting from the primary mission. This thesis investigates, measures, and validates the effectiveness of a framework to provide additional information to an operator in an augmented reality format. This thesis focuses on applications of heads-up displays for rotorcraft pilots. Virtual reality (VR) environments are augmented to accept externally computed situational awareness information using established frameworks for human systems engineering. These frameworks will ensure that such additional information will not negatively affect the operator\u27s cognition in performing mission-critical tasks. The research work described in this thesis will demonstrate that such augmented heads-up displays will provide civilian and military actors with enhanced tools for operational effectiveness, safety and survivability especially in critical situations

Aerospace medicine and biology: A continuing bibliography with indexes (supplement 359)

This bibliography lists 164 reports, articles and other documents introduced into the NASA Scientific and Technical Information System during Jan. 1992. Subject coverage includes: aerospace medicine and physiology, life support systems and man/system technology, protective clothing, exobiology and extraterrestrial life, planetary biology, and flight crew behavior and performance

Situation awareness measurement: A review of applicability for C4i environments

The construct of situation awareness (SA) has become a core theme within the human factors (HF) research community. Consequently, there have been numerous attempts to develop reliable and valid measures of SA but there is a lack of techniques developed specifically for the assessment of SA in command, control, communication, computers and intelligence (C4i) environments. During the design, development and evaluation of novel systems, technology and procedures, valid and reliable situation awareness measurement techniques are required for the assessment of individual and team SA, in order to determine the improvements (or in some cases decrements) resulting from proposed design and technological interventions. The paper presents a review of existing situation awareness measurement techniques for their suitability for use in the assessment of SA in C4i environments. Seventeen SA measures were evaluated against a set of HF methods criteria. It was concluded that current SA measurement techniques are inadequate by themselves for use in the assessment of SA in C4i environments, and a multiple-measure approach utilising different approaches is recommended

Enhancing General Aviation Aircraft Safety With Supplemental Angle Of Attack Systems

Between 2001 and 2010, the Federal Aviation Administration determined 40.2 percent of fatal general aviation accidents in the United States, or 1,259 accidents, were caused by inflight loss of control. General aviation accidents continue to be responsible for more than 440 fatalities each year in the United States, and approximately 40 percent of these are caused by loss of control, mainly stalls. This sequential mixed methods study tested the theory that the number of stalls in the traffic pattern in light general aviation aircraft can be reduced when aircraft are equipped with supplemental angle of attack instrumentation designed to provide the pilot continuous situational awareness regarding remaining lift available for the current aircraft configuration and flight conditions. Quantitative research questions first addressed the relationship between stabilized approaches and installation of supplemental AOA systems through multiple regressions. Safety surveys of flight instructors and students were then used to probe significant findings regarding AOA system contributions to flying stabilized approaches. These follow up surveys were designed to better understand the quantitative results as well as collect information useful to developing future training. Over the course of 1,616 analyzed approaches flown between October 1, 2013 and December 31, 2014, the addition of supplemental angle of attack systems alone did not significantly increase the likelihood of subject pilots flying a stabilized approach. The overall regression models for airspeed and altitude elements of stabilized approaches were significant, but no significant effect of supplemental AOA systems was observed. Likewise, checking each individual AOA system for influence on approach performance against the control group of unmodified aircraft yielded no significant effects. Technical limitations of flight data collection equipment and lack of formal training for subject pilots were identified as possible masks of AOA system effects. Recommendations for formal training and future research are made based on these limitations