Situation awareness measurement: A review of applicability for C4i environments

The construct of situation awareness (SA) has become a core theme within the human factors (HF) research community. Consequently, there have been numerous attempts to develop reliable and valid measures of SA but there is a lack of techniques developed specifically for the assessment of SA in command, control, communication, computers and intelligence (C4i) environments. During the design, development and evaluation of novel systems, technology and procedures, valid and reliable situation awareness measurement techniques are required for the assessment of individual and team SA, in order to determine the improvements (or in some cases decrements) resulting from proposed design and technological interventions. The paper presents a review of existing situation awareness measurement techniques for their suitability for use in the assessment of SA in C4i environments. Seventeen SA measures were evaluated against a set of HF methods criteria. It was concluded that current SA measurement techniques are inadequate by themselves for use in the assessment of SA in C4i environments, and a multiple-measure approach utilising different approaches is recommended

Measuring situation awareness in complex systems: Comparison of measures study

Situation Awareness (SA) is a distinct critical commodity for teams working in complex industrial systems and its measurement is a key provision in system, procedural and training design efforts. This article describes a study that was undertaken in order to compare three different SA measures (a freeze probe recall approach, a post trial subjective rating approach and a critical incident interview technique) when used to assess participant SA during a military planning task. The results indicate that only the freeze probe recall method produced a statistically significant correlation with performance on the planning task and also that there was no significant correlation between the three methods, which suggests that they were effectively measuring different things during the trials. In conclusion, the findings, whilst raising doubts over the validity of post trial subjective rating and interview-based approaches, offer validation evidence for the use of freeze probe recall approaches to measure SA. The findings are subsequently discussed with regard to their implications for the future measurement of SA in complex collaborative systems

Pilots’ visual scan pattern and attention distribution during the pursuit of a dynamic target

Introduction: The current research is investigating pilots’ visual scan patterns in order to assess attention distribution during air-to-air manoeuvers. Method: A total of thirty qualified mission-ready fighter pilots participated in this research. Eye movement data were collected by a portable head-mounted eye-tracking device, combined with a jet fighter simulator. To complete the task, pilots have to search for, pursue, and lock-on a moving target whilst performing air-to-air tasks. Results: There were significant differences in pilots’ saccade duration (msec) in three operating phases including searching (M=241, SD=332), pursuing (M=311, SD=392), and lock-on (M=191, SD=226). Also, there were significant differences in pilots’ pupil sizes (pixel2) of which lock-on phase was the largest (M=27237, SD=6457), followed by pursuing (M=26232, SD=6070), then searching (M=25858, SD=6137). Furthermore, there were significant differences between expert and novice pilots on the percentage of fixation on the HUD, time spent looking outside the cockpit, and the performance of situational awareness (SA). Discussion: Experienced pilots have better SA performance and paid more attention to the HUD but focused less outside the cockpit when compared with novice pilots. Furthermore, pilots with better SA performance exhibited a smaller pupil size during the operational phase of lock-on whilst pursuing a dynamic target. Understanding pilots’ visual scan patterns and attention distribution are beneficial to the design of interface displays in the cockpit and in developing human factors training syllabi to improve safety of flight operations

Situational awareness and safety

This paper considers the applicability of situation awareness concepts to safety in the control of complex systems. Much of the research to date has been conducted in aviation, which has obvious safety implications. It is argued that the concepts could be extended to other safety critical domains. The paper presents three theories of situational awareness: the three-level model, the interactive sub-systems approach, and the perceptual cycle. The difference between these theories is the extent to which they emphasise process or product as indicative of situational awareness. Some data from other studies are discussed to consider the negative effects of losing situational awareness, as this has serious safety implications. Finally, the application of situational awareness to system design, and training are presented

Pilots’ visual scan pattern and situation awareness in flight operations

Introduction: Situation awareness (SA) is considered an essential prerequisite for safe flying. If the impact of visual scanning patterns on a pilot’s situation awareness could be identified in flight operations, then eye-tracking tools could be integrated with flight simulators to improve training efficiency. Method: Participating in this research were 18 qualified, mission-ready fighter pilots. The equipment included high-fidelity and fixed-base type flight simulators and mobile head-mounted eye-tracking devices to record a subject’s eye movements and SA while performing air-to-surface tasks. Results: There were significant differences in pilots’ percentage of fixation in three operating phases: preparation (M = 46.09, SD = 14.79), aiming (M = 24.24, SD = 11.03), and release and break-away (M = 33.98, SD = 14.46). Also, there were significant differences in pilots’ pupil sizes, which were largest in the aiming phase (M = 27,621, SD = 6390.8), followed by release and break-away (M = 27,173, SD = 5830.46), then preparation (M = 25,710, SD = 6078.79), which was the smallest. Furthermore, pilots with better SA performance showed lower perceived workload (M = 30.60, SD = 17.86), and pilots with poor SA performance showed higher perceived workload (M = 60.77, SD = 12.72). Pilots’ percentage of fixation and average fixation duration among five different areas of interest showed significant differences as well. Discussion: Eye-tracking devices can aid in capturing pilots’ visual scan patterns and SA performance, unlike traditional flight simulators. Therefore, integrating eye-tracking devices into the simulator may be a useful method for promoting SA training in flight operations, and can provide in-depth understanding of the mechanism of visual scan patterns and information processing to improve training effectiveness in aviation

The evaluation of pilots performance and mental workload by eye movement

Pilots make important decisions often using ambiguous information, while under stresses and with very little time. During flight operations detecting the warning light of system failure is a task with real-world application relates to measurement of pilot's performance and eye movement. The demand for a pilot’s visual and situational awareness in multiple tasks can be detrimental during pilots’ mental overload conditions. The purpose of this research is to evaluate the relationship between pilot’s mental workload and operational performance by eye tracking. Collecting eye movement data during flight operations in a virtual reality of flight simulator provided useful information to analysis participants’ cognitive processes. There were 36 pilots participated in this research, the experience of flight hours between 320 and 2,920, the range of age between 26 and 51 years old. The apparatus included Applied Science Laboratories (ASL) eye tracking, IDF flight simulator and NASA_TLX for data collection. The results show that pilots with high SA detecting hydraulic malfunction have shorter total fixation duration on Air Speed Indicator and longer total fixation duration on Altitude Indicator, Vertical Speed Indicator, Right multi-display and Left multi-display compared with pilots without detecting the signal of hydraulic malfunction. Pilots’ total fixation time on Integration Control Panel, Altitude Indicator, Attitude Indicator and Right Multi-display, and pilots’ subjective rating on NASA-TLX effort dimension for the mission of close pattern have significant relationship with pilots’ performance on the operational time for completing the tactic mission. Experienced pilots operate aircraft familiar with monitoring Airspeed Indicator and kinetic maneuvering result in less fuel consumption. This study could provide guidelines for future training design to reduce pilots mental workload and improve situational awareness for enhancing flight safety

Best Practices for Evaluating Flight Deck Interfaces for Transport Category Aircraft with Particular Relevance to Issues of Attention, Awareness, and Understanding CAST SE-210 Output 2 Report 6 of 6

Attention, awareness, and understanding of the flight crew are a critical contributor to safety and the flight deck plays a critical role in supporting these cognitive functions. Changes to the flight deck need to be evaluated for whether the changed device provides adequate support for these functions. This report describes a set of diverse evaluation methods. The report recommends designing the interface-evaluation to span the phases of the device development, from early to late, and it provides methods appropriate at each phase. It describes the various ways in which an interface or interface component can fail to support awareness as potential issues to be assessed in evaluation. It summarizes appropriate methods to evaluate different issues concerning inadequate support for these functions, throughout the phases of development

Report of the workshop on Aviation Safety/Automation Program

As part of NASA's responsibility to encourage and facilitate active exchange of information and ideas among members of the aviation community, an Aviation Safety/Automation workshop was organized and sponsored by the Flight Management Division of NASA Langley Research Center. The one-day workshop was held on October 10, 1989, at the Sheraton Beach Inn and Conference Center in Virginia Beach, Virginia. Participants were invited from industry, government, and universities to discuss critical questions and issues concerning the rapid introduction and utilization of advanced computer-based technology into the flight deck and air traffic controller workstation environments. The workshop was attended by approximately 30 discipline experts, automation and human factors researchers, and research and development managers. The goal of the workshop was to address major issues identified by the NASA Aviation Safety/Automation Program. Here, the results of the workshop are documented. The ideas, thoughts, and concepts were developed by the workshop participants. The findings, however, have been synthesized into a final report primarily by the NASA researchers