Human Factors and Neurophysiological Metrics in Air Traffic Control: a Critical Review

International audienceThis article provides the reader a focused and organised review of the research progresses on neurophysiological indicators, also called “neurometrics”, to show how neurometrics could effectively address some of the most important Human Factors (HFs) needs in the Air Traffic Management (ATM) field. The state of the art on the most involved HFs and related cognitive processes (e.g. mental workload, cognitive training) is presented together with examples of possible applications in the current and future ATM scenarios, in order to better understand and highlight the available opportunities of such neuroscientific applications. Furthermore, the paper will discuss the potential enhancement that further research and development activities could bring to the efficiency and safety of the ATM service

How neurophysiological measures can be used to enhance the evaluation of remote tower solutions

International audienceNew solutions in operational environments are often, among objective measurements, evaluated by using subjective assessment and judgement from experts. Anyhow, it has been demonstrated that subjective measures suffer from poor resolution due to a high intra and inter operator variability. Also, performance measures, if available, could provide just partial information, since an operator could achieve the same performance but experiencing a different workload. In this study we aimed to demonstrate i) the higher resolution of neurophysiological measures in comparison to subjective ones, and ii) how the simultaneous employment of neurophysiological measures and behavioural ones could allow a holistic assessment of operational tools. In this regard, we tested the effectiveness of an EEG-based neurophysiological index (WEEG index) in comparing two different solutions (i.e. Normal and Augmented) in terms of experienced workload. In this regard, 16 professional Air Traffic Controllers (ATCOs) have been asked to perform two operational scenarios. Galvanic Skin Response (GSR) has also been recorded to evaluate the level of arousal (i.e. operator involvement) during the two scenarios execution. NASA-TLX questionnaire has been used to evaluate the perceived workload, and an expert was asked to assess performance achieved by the ATCOs. Finally, reaction times on specific operational events relevant for the assessment of the two solutions, have also been collected. Results highlighted that the Augmented solution induced a local increase in subjects performance (Reaction times). At the same time, this solution induced an increase in the workload experienced by the participants (WEEG). Anyhow, this increase is still acceptable, since it did not negatively impact the performance and has to be intended only as a consequence of the higher engagement of the ATCOs. This behavioural effect is totally in line with physiological results obtained in terms of arousal (GSR), that increased during the scenario with augmentation. Subjective measures (NASA-TLX) did not highlight any significant variation in perceived workload. These results suggest that neurophysiological measure provide additional information than behavioural and subjective ones, even at a level of few seconds, and its employment during the pre-operational activities (e.g. design process) could allow a more holistic and accurate evaluation of new solutions

Neurophysiological vigilance characterisation and assessment: Laboratory and realistic validations involving professional air traffic controllers

Vigilance degradation usually causes significant performance decrement. It is also considered the major factor causing the out-of-the-loop phenomenon (OOTL) occurrence. OOTL is strongly related to a high level of automation in operative contexts such as the Air Traffic Management (ATM), and it could lead to a negative impact on the Air Traffic Controllers’ (ATCOs) engagement. As a consequence, being able to monitor the ATCOs’ vigilance would be very important to prevent risky situations. In this context, the present study aimed to characterise and assess the vigilance level by using electroencephalographic (EEG) measures. The first study, involving 13 participants in laboratory settings allowed to find out the neurophysiological features mostly related to vigilance decrements. Those results were also confirmed under realistic ATM settings recruiting 10 professional ATCOs. The results demonstrated that (i) there was a significant performance decrement related to vigilance reduction; (ii) there were no substantial differences between the identified neurophysiological features in controlled and ecological settings, and the EEG-channel configuration defined in laboratory was able to discriminate and classify vigilance changes in ATCOs’ vigilance with high accuracy (up to 84%); (iii) the derived two EEG-channel configuration was able to assess vigilance variations reporting only slight accuracy reduction

Development of a System for the Training Assessment and Mental Workload Evaluation

Several studies have demonstrated that the main cause of accidents are due to Human Factor (HF) failures. Humans are the least and last controllable factor in the activity workflows, and the availability of tools able to provide objective information about the user’s cognitive state should be very helpful in maintain proper levels of safety. To overcome these issues, the objectives of the PhD covered three topics. The first phase was focused on the study of machine-learning techniques to evaluate the user’s mental workload during the execution of a task. In particular, the methodology was developed to address two important limitations: i) over-time reliability (no recalibration of the algorithm); ii) automatic brain features selection to avoid both the underfitting and overfitting problems. The second phase was dedicated to the study of the training assessment. In fact, the standard training evaluation methods do not provide any objective information about the amount of brain activation\resources required by the user, neither during the execution of the task, nor across the training sessions. Therefore, the aim of this phase was to define a neurophysiological methodology able to address such limitation. The third phase of the PhD consisted in overcoming the lack of neurophysiological studies regarding the evaluation of the cognitive control behaviour under which the user performs a task. The model introduced by Rasmussen was selected to seek neurometrics to characterize the skill, rule and knowledge behaviours by means of the user’s brain activity. The experiments were initially ran in controlled environments, whilst the final tests were carried out in realistic environments. The results demonstrated the validity of the developed algorithm and methodologies (2 patents pending) in solving the issues quoted initially. In addition, such results brought to the submission of a H2020-SMEINST project, for the realization of a device based on such results

Behavioural Markers of Air Traffic Controller Development

A key challenge when introducing new systems and technologies into Air Traffic control (ATC) is to understand levels of emerging controller proficiency ahead of scheduled implementation. Behavioural markers have been used in several complex industries to assess levels of non-technical skill; however these measures invariably focus upon the desired behaviours attained by the end of training. This research has explored how an Air Traffic Controller’s (ATCO’s) overt non-technical behaviour changes in presence and prevalence as they progress their expertise during training. Through document review, expert engagement, and most extensively direct observation of ATCOs during and after training, a number of non-technical behaviours indicative of varying proficiency have been identified. These markers were placed within a simple three-level learning and development framework. Five categories emerged across the behaviours identified; i) input and interaction with the Human Machine Interface (HMI), ii) interaction with others, iii) physical posture and body Language, iv) attitude and mood; v) communications and verbal commentary. An observation sheet containing the markers was iteratively developed, tested, and refined in various ATC environments. Both expert ATCOs undergoing system transition training, and ab-initio trainee controllers undertaking aerodrome training were followed through longitudinal study. A capped frequency count was used to record the precise presence of individual markers. Several dual-observations were also undertaken to determine inter-rater reliability and construct validity. In total, the performance of the individual markers has been evaluated across 129 real-world observations. 30 markers demonstrate reliable correlations for changing prevalence against total system exposure time and provide an original means of tracking and monitoring subtle changes in the behaviour of ATCOs, as their levels of proficiency in the task matures with new ATC systems

Encoding decisions and expertise in the operator's eyes: Using eye-tracking as input for system adaptation

We investigated the possibility of developing a decision support system (DSS) that integrates eye-fixation measurements to better adapt its suggestions. Indeed, eye fixation give insight into human decision-making: Individuals tend to pay more attention to key information in line with their upcoming selection. Thus, eye-fixation measures can help the DSS to better capture the context that determines user decisions. Twenty-two participants performed a simplified Air Traffic Control (ATC) simulation in which they had to decide to accept or to modify route suggestions according to specific parameter values displayed on the screen. Decisions and fixation times on each parameter were recorded. The user fixation times were used by an algorithm to estimate the utility of each parameter for its decision. Immediately after this training phase, the algorithm generated new route suggestions under two conditions: 1) Taking into account the participant's decisions, 2) Taking into account the participant's decisions plus their visual behavior using the measurements of dwell times on displayed parameters. Results showed that system suggestions were more accurate than the base system when taking into account the participant's decisions, and even more accurate using their dwell times. Capturing the crucial information for the decision using the eye tracker accelerated the DSS learning phase, and thus helped to further enhance the accuracy of consecutive suggestions. Moreover, exploratory eye-tracking analysis reflected two different stages of the decision-making process, with longer dwell times on relevant parameters (i.e. involved in a rule) during the entire decision time course, and frequency of fixations on these relevant parameters that increased, especially during the last fixations prior to the decision. Consequently, future DSS integrating eye-tracking data should pay specific care to the final fixations prior to the decision. In general, our results emphasize the potential interest of eye-tracking to enhance and accelerate system adaptation to user preference, knowledge, and expertise

Innovative Man Machine Interfaces In Aeronautics

The research activity focused on the study, design and evaluation of innovative human-machine interfaces based on virtual three-dimensional environments. It is based on the brain electrical activities recorded in real time through the electrical impulses emitted by the brain waves of the user. The achieved target is to identify and sort in real time the different brain states and adapt the interface and/or stimuli to the corresponding emotional state of the user. The setup of an experimental facility based on an innovative experimental methodology for “man in the loop" simulation was established. It allowed involving during pilot training in virtually simulated flights, both pilot and flight examiner, in order to compare the subjective evaluations of this latter to the objective measurements of the brain activity of the pilot. This was done recording all the relevant information versus a time-line. Different combinations of emotional intensities obtained, led to an evaluation of the current situational awareness of the user. These results have a great implication in the current training methodology of the pilots, and its use could be extended as a tool that can improve the evaluation of a pilot/crew performance in interacting with the aircraft when performing tasks and procedures, especially in critical situations. This research also resulted in the design of an interface that adapts the control of the machine to the situation awareness of the user. The new concept worked on, aimed at improving the efficiency between a user and the interface, and gaining capacity by reducing the user’s workload and hence improving the system overall safety. This innovative research combining emotions measured through electroencephalography resulted in a human-machine interface that would have three aeronautical related applications: • An evaluation tool during the pilot training; • An input for cockpit environment; • An adaptation tool of the cockpit automation

Foundations and applications of human-machine interaction

Zugleich gedruckt erschienen im Universitätsverlag der TU Berlin unter der ISBN 978-3-7983-2624-8.Der vorliegende Abstractband zur 10 Berliner Werkstatt MMS gibt einen Einblick in die aktuelle Forschung im Bereich der Mensch-Maschine-Interaktion. Einen besonderen Fokus stellt das Wechselspiel von Grundlagenforschung und anwendungsbezogener Forschung dar, was sich im breiten Themenspektrum widerspiegelt, welches von theoretischen und methodischen Betrachtungen bis hin zu anwendungsnahen Fragestellungen reicht. Dabei finden Inhalte aus allen Phasen des Forschungsprozesses Beachtung, sodass auch im Rahmen der 10. Berliner Werkstatt MMS wieder sowohl neue Untersuchungskonzepte als auch abschließende Befunde diskutiert werden. Zentrale Themengebiete sind u. a. Fahrer-Fahrzeug Interaktion, Assistenzsysteme, User Experience, Usability, Ubiquitous Computing, Mixed & Virtual Reality, Robotics & Automation, Wahrnehmungsspezifika sowie Psychophysiologie und Beanspruchung in der Mensch-Maschine-Interaktion.The abstracts of the 10th Berlin Workshop Human-Machine Systems provide an insight into the current research in the field of human-machine interaction. The main focus lies on the interplay between basic and applied research, which is reflected in the wide range of subjects: from theoretical and methodological issues to application oriented considerations. Again all stages of the research process are represented in the contributions of the 10th Berlin Workshop HMS. This means new research concepts as well as final results are subject of this volume. Central topics include driver-vehicle interaction, assistance systems, user experience, usability, ubiquitous computing, mixed and virtual reality, robotics & automation, perception specifics, as well as psychophysiology and workload in human-machine interaction

Assuring safe and efficient operation of UAV using explainable machine learning

The accurate estimation of airspace capacity in unmanned traffic management (UTM) operations is critical for a safe, efficient, and equitable allocation of airspace system resources. While conventional approaches for assessing airspace complexity certainly exist, these methods fail to capture true airspace capacity, since they fail to address several important variables (such as weather). Meanwhile, existing AI-based decision-support systems evince opacity and inexplicability, and this restricts their practical application. With these challenges in mind, the authors propose a tailored solution to the needs of demand and capacity management (DCM) services. This solution, by deploying a synthesized fuzzy rule-based model and deep learning will address the trade-off between explicability and performance. In doing so, it will generate an intelligent system that will be explicable and reasonably comprehensible. The results show that this advisory system will be able to indicate the most appropriate regions for unmanned aerial vehicle (UAVs) operation, and it will also increase UTM airspace availability by more than 23%. Moreover, the proposed system demonstrates a maximum capacity gain of 65% and a minimum safety gain of 35%, while possessing an explainability attribute of 70%. This will assist UTM authorities through more effective airspace capacity estimation and the formulation of new operational regulations and performance requirements