Petri net-based modelling of human–automation conflicts in aviation

Analyses of aviation safety reports reveal that human–machine conflicts induced by poor automation design are remarkable precursors of accidents. A review of different crew–automation conflicting scenarios shows that they have a common denominator: the autopilot behaviour interferes with the pilot's goal regarding the flight guidance via ‘hidden’ mode transitions. Considering both the human operator and the machine (i.e. the autopilot or the decision functions) as agents, we propose a Petri net model of those conflicting interactions, which allows them to be detected as deadlocks in the Petri net. In order to test our Petri net model, we designed an autoflight system that was formally analysed to detect conflicting situations. We identified three conflicting situations that were integrated in an experimental scenario in a flight simulator with 10 general aviation pilots. The results showed that the conflicts that we had a-priori identified as critical had impacted the pilots' performance. Indeed, the first conflict remained unnoticed by eight participants and led to a potential collision with another aircraft. The second conflict was detected by all the participants but three of them did not manage the situation correctly. The last conflict was also detected by all the participants but provoked typical automation surprise situation as only one declared that he had understood the autopilot behaviour. These behavioural results are discussed in terms of workload and number of fired ‘hidden’ transitions. Eventually, this study reveals that both formal and experimental approaches are complementary to identify and assess the criticality of human–automation conflicts. Practitioner Summary: We propose a Petri net model of human–automation conflicts. An experiment was conducted with general aviation pilots performing a scenario involving three conflicting situations to test the soundness of our formal approach. This study reveals that both formal and experimental approaches are complementary to identify and assess the criticality conflicts

What the heck is it doing? Better understanding human-machine conflicts through models

This paper deals with human-machine conflicts with a special focus on conflicts caused by an “automation surprise”. Considering both the human operator and the machine autopilot or decision functions as agents, we propose Petri net based models of two real cases and we show how modelling each agent’s possible actions is likely to highlight conflict states as deadlocks in the Petri net. A general conflict model is then be proposed and paves the way for further on-line human-machine conflict forecast and detection

Safety‐oriented discrete event model for airport A‐SMGCS reliability assessment

A detailed analysis of State of the Art Technologies and Procedures into Airport Advanced-Surface Movement Guidance and Control Systems has been provided in this thesis, together with the review ofStatistical Monte Carlo Analysis, Reliability Assessment and Petri Nets theories. This practical and theoretical background has lead the author to the conclusion that there is a lack of linkage in between these fields. At the same of time the rapid increasing of Air Traffic all over the world, has brought in evidence the urgent need of practical instruments able to identify and quantify the risks connected with Aircraft operations on the ground, since the Airport has shown to be the actual ‘bottle neck’ of the entire Air Transport System. Therefore, the only winning approach to such a critical matter has to be multi-disciplinary, sewing together apparently different subjects, coming from the most disparate areas of interest and trying to fulfil the gap. The result of this thesis work has come to a start towards the end, when a Timed Coloured Petri Net (TCPN) model of a ‘sample’ Airport A-SMGCS has been developed, that is capable of taking into account different orders of questions arisen during these recent years and tries to give them some good answers. The A-SMGCS Airport model is, in the end, a parametric tool relying on Discrete Event System theory, able to perform a Reliability Analysis of the system itself, that: • uses a Monte Carlo Analysis applied to a Timed Coloured Petri Net, whose purpose is to evaluate the Safety Level of Surface Movements along an Airport • lets the user to analyse the impact of Procedures and Reliability Indexes of Systems such as Surface Movement Radars, Automatic Dependent Surveillance-Broadcast, Airport Lighting Systems, Microwave Sensors, and so on… onto the Safety Level of Airport Aircraft Transport System • not only is a valid instrument in the Design Phase, but it is useful also into the Certifying Activities an in monitoring the Safety Level of the above mentioned System with respect to changes to Technologies and different Procedures.This TCPN model has been verified against qualitative engineering expectations by using simulation experiments and occupancy time schedules generated a priori. Simulation times are good, and since the model has been written into Simulink/Stateflow programming language, it can be compiled to run real-time in C language (Real-time workshop and Stateflow Coder), thus relying on portable code, able to run virtually on any platform, giving even better performances in terms of execution time. One of the most interesting applications of this work is the estimate, for an Airport, of the kind of A-SMGCS level of implementation needed (Technical/Economical convenience evaluation). As a matter of fact, starting from the Traffic Volume and choosing the kind of Ground Equipment to be installed, one can make predictions about the Safety Level of the System: if the value is compliant with the TLS required by ICAO, the A-SMGCS level of Implementation is sufficiently adequate. Nevertheless, even if the Level of Safety has been satisfied, some delays due to reduced or simplified performances (even if Safety is compliant) of some of the equipment (e.g. with reference to False Alarm Rates) can lead to previously unexpected economical consequences, thus requiring more accurate systems to be installed, in order to meet also Airport economical constraints. Work in progress includes the analysis of the effect of weather conditions and re-sequencing of a given schedule. The effect of re-sequencing a given schedule is not yet enough realistic since the model does not apply inter arrival and departure separations. However, the model might show some effect on different sequences based on runway occupancy times. A further developed model containing wake turbulence separation conditions would be more sensitive for this case. Hence, further work will be directed towards: • The development of On-Line Re-Scheduling based on the available actual runway/taxiway configuration and weather conditions. • The Engineering Safety Assessment of some small Italian Airport A-SMGCSs (Model validation with real data). • The application of Stochastic Differential Equations systems in order to evaluate the collision risk on the ground inside the Place alone on the Petri Net, in the event of a Short Term Conflict Alert (STCA), by adopting Reich Collision Risk Model. • Optimal Air Traffic Control Algorithms Synthesis (Adaptive look-ahead Optimization), by Dynamically Timed Coloured Petri Nets, together with the implementation of Error-Recovery Strategies and Diagnosis Functions

Sequential Monte Carlo simulation of collision risk in free flight air traffic

Within HYBRIDGE a novel approach in speeding up Monte Carlo simulation of rare events has been developed. In the current report this method is extended for application to simulating collisions with a stochastic dynamical model of an air traffic operational concept. Subsequently this extended Monte Carlo simulation approach is applied to a simulation model of an advanced free flight operational concept; i.e. one in which aircraft are responsible for self separation with each other. The Monte Carlo simulation results obtained for this advanced concept show that the novel method works well, and that it allows studying rare events that stayed invisible in previous Monte Carlo simulations of advanced air traffic operational concepts

"Automation Surprise" in Aviation

Conflicts between the pilot and the automation, when pilots detect but do not understand them, cause “automation sur- prise” situations and jeopardize flight safety. We conducted an experiment in a 3-axis motion flight simulator with 16 pi- lots equipped with an eye-tracker to analyze their behavior and eye movements during the occurrence of such a situation. The results revealed that this conflict engages participant’s at- tentional abilities resulting in excessive and inefficient visual search patterns. This experiment confirmed the crucial need to design solutions for detecting the occurrence of conflict- ual situations and to assist the pilots. We therefore proposed an approach to formally identify the occurrence of “automa- tion surprise” conflicts based on the analysis of “silent mode changes” of the autopilot. A demonstrator was implemented and allowed for the automatic trigger of messages in the cock- pit that explains the autopilot behavior. We implemented a real-time demonstrator that was tested as a proof-of-concept with 7 subjects facing 3 different conflicts with automation. The results shown the efficacy of this approach which could be implemented in existing cockpits

Existing and Required Modeling Capabilities for Evaluating ATM Systems and Concepts

ATM systems throughout the world are entering a period of major transition and change. The combination of important technological developments and of the globalization of the air transportation industry has necessitated a reexamination of some of the fundamental premises of existing Air Traffic Management (ATM) concepts. New ATM concepts have to be examined, concepts that may place more emphasis on: strategic traffic management; planning and control; partial decentralization of decision-making; and added reliance on the aircraft to carry out strategic ATM plans, with ground controllers confined primarily to a monitoring and supervisory role. 'Free Flight' is a case in point. In order to study, evaluate and validate such new concepts, the ATM community will have to rely heavily on models and computer-based tools/utilities, covering a wide range of issues and metrics related to safety, capacity and efficiency. The state of the art in such modeling support is adequate in some respects, but clearly deficient in others. It is the objective of this study to assist in: (1) assessing the strengths and weaknesses of existing fast-time models and tools for the study of ATM systems and concepts and (2) identifying and prioritizing the requirements for the development of additional modeling capabilities in the near future. A three-stage process has been followed to this purpose: 1. Through the analysis of two case studies involving future ATM system scenarios, as well as through expert assessment, modeling capabilities and supporting tools needed for testing and validating future ATM systems and concepts were identified and described. 2. Existing fast-time ATM models and support tools were reviewed and assessed with regard to the degree to which they offer the capabilities identified under Step 1. 3 . The findings of 1 and 2 were combined to draw conclusions about (1) the best capabilities currently existing, (2) the types of concept testing and validation that can be carried out reliably with such existing capabilities and (3) the currently unavailable modeling capabilities that should receive high priority for near-term research and development. It should be emphasized that the study is concerned only with the class of 'fast time' analytical and simulation models. 'Real time' models, that typically involve humans-in-the-loop, comprise another extensive class which is not addressed in this report. However, the relationship between some of the fast-time models reviewed and a few well-known real-time models is identified in several parts of this report and the potential benefits from the combined use of these two classes of models-a very important subject-are discussed in chapters 4 and 7

Empirical exploration of air traffic and human dynamics in terminal airspaces

Air traffic is widely known as a complex, task-critical techno-social system, with numerous interactions between airspace, procedures, aircraft and air traffic controllers. In order to develop and deploy high-level operational concepts and automation systems scientifically and effectively, it is essential to conduct an in-depth investigation on the intrinsic traffic-human dynamics and characteristics, which is not widely seen in the literature. To fill this gap, we propose a multi-layer network to model and analyze air traffic systems. A Route-based Airspace Network (RAN) and Flight Trajectory Network (FTN) encapsulate critical physical and operational characteristics; an Integrated Flow-Driven Network (IFDN) and Interrelated Conflict-Communication Network (ICCN) are formulated to represent air traffic flow transmissions and intervention from air traffic controllers, respectively. Furthermore, a set of analytical metrics including network variables, complex network attributes, controllers' cognitive complexity, and chaotic metrics are introduced and applied in a case study of Guangzhou terminal airspace. Empirical results show the existence of fundamental diagram and macroscopic fundamental diagram at the route, sector and terminal levels. Moreover, the dynamics and underlying mechanisms of "ATCOs-flow" interactions are revealed and interpreted by adaptive meta-cognition strategies based on network analysis of the ICCN. Finally, at the system level, chaos is identified in conflict system and human behavioral system when traffic switch to the semi-stable or congested phase. This study offers analytical tools for understanding the complex human-flow interactions at potentially a broad range of air traffic systems, and underpins future developments and automation of intelligent air traffic management systems.Comment: 30 pages, 28 figures, currently under revie

Modeling and control of operator functional state in a unified framework of fuzzy inference petri nets

Background and objective: In human-machine (HM) hybrid control systems, human operator and machine cooperate to achieve the control objectives. To enhance the overall HM system performance, the discrete manual control task-load by the operator must be dynamically allocated in accordance with continuous-time fluctuation of psychophysiological functional status of the operator, so-called operator functional state (OFS). The behavior of the HM system is hybrid in nature due to the co-existence of discrete task-load (control) variable and continuous operator performance (system output) variable. Methods: Petri net is an effective tool for modeling discrete event systems, but for hybrid system involving discrete dynamics, generally Petri net model has to be extended. Instead of using different tools to represent continuous and discrete components of a hybrid system, this paper proposed a method of fuzzy inference Petri nets (FIPN) to represent the HM hybrid system comprising a Mamdani-type fuzzy model of OFS and a logical switching controller in a unified framework, in which the task-load level is dynamically reallocated between the operator and machine based on the model-predicted OFS. Furthermore, this paper used a multi-model approach to predict the operator performance based on three electroencephalographic (EEG) input variables (features) via the Wang-Mendel (WM) fuzzy modeling method. The membership function parameters of fuzzy OFS model for each experimental participant were optimized using artificial bee colony (ABC) evolutionary algorithm. Three performance indices, RMSE, MRE, and EPR, were computed to evaluate the overall modeling accuracy. Results: Experiment data from six participants are analyzed. The results show that the proposed method (FIPN with adaptive task allocation) yields lower breakdown rate (from 14.8% to 3.27%) and higher human performance (from 90.30% to 91.99%). Conclusion: The simulation results of the FIPN-based adaptive HM (AHM) system on six experimental participants demonstrate that the FIPN framework provides an effective way to model and regulate/optimize the OFS in HM hybrid systems composed of continuous-time OFS model and discrete-event switching controller

Safety Sufficiency for NextGen: Assessment of Selected Existing Safety Methods, Tools, Processes, and Regulations

NextGen is a complex socio-technical system and, in many ways, it is expected to be more complex than the current system. It is vital to assess the safety impact of the NextGen elements (technologies, systems, and procedures) in a rigorous and systematic way and to ensure that they do not compromise safety. In this study, the NextGen elements in the form of Operational Improvements (OIs), Enablers, Research Activities, Development Activities, and Policy Issues were identified. The overall hazard situation in NextGen was outlined; a high-level hazard analysis was conducted with respect to multiple elements in a representative NextGen OI known as OI-0349 (Automation Support for Separation Management); and the hazards resulting from the highly dynamic complexity involved in an OI-0349 scenario were illustrated. A selected but representative set of the existing safety methods, tools, processes, and regulations was then reviewed and analyzed regarding whether they are sufficient to assess safety in the elements of that OI and ensure that safety will not be compromised and whether they might incur intolerably high costs