Predicting Pilot Misperception of Runway Excursion Risk Through Machine Learning Algorithms of Recorded Flight Data

The research used predictive models to determine pilot misperception of runway excursion risk associated with unstable approaches. The Federal Aviation Administration defined runway excursion as a veer-off or overrun of the runway surface. The Federal Aviation Administration also defined a stable approach as an aircraft meeting the following criteria: (a) on target approach airspeed, (b) correct attitude, (c) landing configuration, (d) nominal descent angle/rate, and (e) on a straight flight path to the runway touchdown zone. Continuing an unstable approach to landing was defined as Unstable Approach Risk Misperception in this research. A review of the literature revealed that an unstable approach followed by the failure to execute a rejected landing was a common contributing factor in runway excursions. Flight Data Recorder data were archived and made available by the National Aeronautics and Space Administration for public use. These data were collected over a four-year period from the flight data recorders of a fleet of 35 regional jets operating in the National Airspace System. The archived data were processed and explored for evidence of unstable approaches and to determine whether or not a rejected landing was executed. Once identified, those data revealing evidence of unstable approaches were processed for the purposes of building predictive models. SAS™ Enterprise MinerR was used to explore the data, as well as to build and assess predictive models. The advanced machine learning algorithms utilized included: (a) support vector machine, (b) random forest, (c) gradient boosting, (d) decision tree, (e) logistic regression, and (f) neural network. The models were evaluated and compared to determine the best prediction model. Based on the model comparison, the decision tree model was determined to have the highest predictive value. The Flight Data Recorder data were then analyzed to determine predictive accuracy of the target variable and to determine important predictors of the target variable, Unstable Approach Risk Misperception. Results of the study indicated that the predictive accuracy of the best performing model, decision tree, was 99%. Findings indicated that six variables stood out in the prediction of Unstable Approach Risk Misperception: (1) glideslope deviation, (2) selected approach speed deviation (3) localizer deviation, (4) flaps not extended, (5) drift angle, and (6) approach speed deviation. These variables were listed in order of importance based on results of the decision tree predictive model analysis. The results of the study are of interest to aviation researchers as well as airline pilot training managers. It is suggested that the ability to predict the probability of pilot misperception of runway excursion risk could influence the development of new pilot simulator training scenarios and strategies. The research aids avionics providers in the development of predictive runway excursion alerting display technologies

Classification and reduction of pilot error

Human error is a primary or contributing factor in about two-thirds of commercial aviation accidents worldwide. With the ultimate goal of reducing pilot error accidents, this contract effort is aimed at understanding the factors underlying error events and reducing the probability of certain types of errors by modifying underlying factors such as flight deck design and procedures. A review of the literature relevant to error classification was conducted. Classification includes categorizing types of errors, the information processing mechanisms and factors underlying them, and identifying factor-mechanism-error relationships. The classification scheme developed by Jens Rasmussen was adopted because it provided a comprehensive yet basic error classification shell or structure that could easily accommodate addition of details on domain-specific factors. For these purposes, factors specific to the aviation environment were incorporated. Hypotheses concerning the relationship of a small number of underlying factors, information processing mechanisms, and error types types identified in the classification scheme were formulated. ASRS data were reviewed and a simulation experiment was performed to evaluate and quantify the hypotheses

Proceedings, MSVSCC 2013

Proceedings of the 7th Annual Modeling, Simulation & Visualization Student Capstone Conference held on April 11, 2013 at VMASC in Suffolk, Virginia

The role of situation awareness in accidents of large-scale technological systems

© 2015 The Institution of Chemical Engineers. Published by Elsevier B.V. All rights reserved. In the last two decades, several serious accidents at large-scale technological systems that have had grave consequences, such as that at Bhopal, have primarily been attributed to human error. However, further investigations have revealed that humans are not the primary cause of these accidents, but have inherited the problems and difficulties of working with complex systems created by engineers. The operators have to comprehend malfunctions in real time, respond quickly, and make rapid decisions to return operational units to normal conditions, but under these circumstances, the mental workload of operators rises sharply, and a mental workload that is too high increases the rate of error. Therefore, cognivitive human features such as situation awareness (SA) - one of the most important prerequisite for decision-making - should be considered and analyzed appropriately. This paper applys the SA Error Taxonomy methodology to analyze the role of SA in three different accidents: (1) A runaway chemical reaction at Institute, West Virginia killing two employees, injuring eight people, and requiring the evacuation of more than 40,000 residents adjacent to the facility, (2) The ignition of a vapor cloud at Bellwood, Illinois that killed one person, injured two employees, and caused significant business interruption, and (3) An explosion at Ontario, California injuring four workers and caused extensive damage to the facility. In addition, the paper presents certain requirements for cognitive operator support system development and operator training under abnormal situations to promote operators' SA in the process industry

What Happened, and Why: Toward an Understanding of Human Error Based on Automated Analyses of Incident Reports

The objective of the Aviation System Monitoring and Modeling project of NASA's Aviation Safety and Security Program was to develop technologies to enable proactive management of safety risk, which entails identifying the precursor events and conditions that foreshadow most accidents. Information about what happened can be extracted from quantitative data sources, but the experiential account of the incident reporter is the best available source of information about why an incident happened. In Volume I, the concept of the Scenario was introduced as a pragmatic guide for identifying similarities of what happened based on the objective parameters that define the Context and the Outcome of a Scenario. In this Volume II, that study continues into the analyses of the free narratives to gain understanding as to why the incident occurred from the reporter s perspective. While this is just the first experiment, the results of our approach are encouraging and indicate that it will be possible to design an automated analysis process guided by the structure of the Scenario that can achieve the level of consistency and reliability of human analysis of narrative reports

Decision-making in Risk Management

The definition of risk introduced in the ISO 31000 standard of 2009 (2018) is uncertain goal achievement; thus, both negative and positive outcomes can be considered. It also implies that risk is not limited to life and health, but may cover all goals of a company. Risk management thus becomes a question of achieving and optimizing multiple goals. Since safety is but one of several considerations, safety may lose out to other more easily measured objectives of a company, such as economics and compliance with regulatory requirements. Risk analyses have a long history of quantification, a tradition that for various reasons has waned and should be revived if safety goals are to be treated together with other goals of a company. The extended scope affects not only company owners and employees but also neighbors, the local community, and the society at large. The stochastic nature of risk and the considerable time lap between decisions and the multiattributed consequences implies that managing risk is exposed to cognitive biases of many sorts. Risk management should be based on a quantitative approach to risk analysis as a protection against the many cognitive biases likely to be present, and managers should be trained to recognize the most common cognitive biases and decision pitfalls

Technical approaches for measurement of human errors

Human error is a significant contributing factor in a very high proportion of civil transport, general aviation, and rotorcraft accidents. The technical details of a variety of proven approaches for the measurement of human errors in the context of the national airspace system are presented. Unobtrusive measurements suitable for cockpit operations and procedures in part of full mission simulation are emphasized. Procedure, system performance, and human operator centered measurements are discussed as they apply to the manual control, communication, supervisory, and monitoring tasks which are relevant to aviation operations

Modelling airport surface safety: a framework for a holistic airport safety management

Airports are complex systems involving the continuous interaction of human operators with the physical infrastructure, technology and procedures to ensure the safe and efficient conduct of flights. From an operational perspective, airport surface operations (i.e. runway and taxiway operations) require the interaction of five main stakeholders (i.e. crew or pilots, air traffic control, airport operator, ground handling and regulator) both to facilitate the ground movement of aircraft and vehicles, and to maintain the surface in a working condition. The complexity of these operations makes the runway and taxiway system vulnerable and presents a risk of failure with the consequent potential for the occurrence of accidents. Therefore, the development and implementation of an effective Safety Management System (SMS) are required to ensure the highest level of safety for surface operations. A SMS is a systematic approach to managing safety based on the four cornerstones of safety policy and objectives, risk management, assurance, and safety promotion. Although the International Civil Aviation Organisation (ICAO) provides the global legislative framework for SMS, the relevant regulations are still to be established at the national level with the consequence that practical guidance on the development and implementation of SMS is rare, and reliable tools to support SMS are lacking. The consequence of this is that the current approach to surface safety management is piecemeal and not integrated. Typically, a single accident and incident type is investigated from the perspective of an individual stakeholder with the consequence that resulting proposals for safety mitigation measures are biased and limited in terms of their impact. In addition, the industry is characterised by non-standardised data collection and investigation practices, insufficient or missing definitions, differing reporting levels, and a lack of a coherent and standardised structure for efficient coding and analysis of safety data. Since these shortcomings are a major barrier to the required holistic and integrated approach to safety management, this thesis addresses the four cornerstones of SMS and recommends major enhancements. In particular, a framework for a holistic airport surface safety management is proposed. The framework comprises the static airport architecture, a process model of surface operations, the determination of causal factors underlying failure modes of these operations, a macroscopic scenario tool and a functional relationship model. Safety data and other data sources feed the framework and a dedicated data pre-processing strategy ensures its validity. Unlike current airport surface safety management practices, the proposed framework assesses the safety of the operations of all relevant actors. Firstly, the airport architecture is modelled and the physical and functional variability of airports defined. Secondly, a process model of surface operations is developed, which captures the tasks of the stakeholders and their interactions with physical airport surface infrastructure. This model serves as a baseline model and guides the further development of the airport SMS. To manage the safety of surface operations, the causes of accidents and incidents must be identified and their impacts understood. To do so, a reference data set combining twelve databases from airlines, airport operators, Air Navigation Service Providers (ANSPs), ground handling companies and regulators is collected. Prior to its analysis, the data is assessed for its quality, and in particular, for its internal validity (i.e. precision), external validity (i.e. accuracy) and in terms of reporting levels. A novel external data validation framework is developed and each database is rated with a data quality index (DQI). In addition, recommendations for reporting systems and safety policies are given. Subsequently, the data is analysed for causal factors across stakeholders and the contribution of the individual actors are highlighted. For example, the analysis shows that the various stakeholders capture different occurrence types and underlying causal factors, often including information that is of potential use for another party. The analysis is complemented by interviews, observations and statistical analysis, and the results are summarised in a new taxonomy. This taxonomy is applicable to all relevant stakeholders and is recommended for operational safety risk management. After the airport surface operations have been modelled and the drivers to safety identified, the results are combined, resulting in a macroscopic scenario tool which supports the management of change (i.e. safety assurance), training and education, and safety communication (i.e. safety promotion) functions of the SMS. Finally, a structured framework to assess the functional relationship between airport surface accidents / incidents and their underlying causal factors is proposed and the system is quantified in terms of safety. Compared to the state-of-the-art safety assessments that are biased and limited in terms of their impact, the holistic approach to surface safety allows modelling the safety impact of each system component, their interactions and the entire airport surface system architecture. The framework for a holistic airport surface safety management developed in this thesis delivers a SMS standard for airports. The standard exceeds international requirements by standardizing the two SMS core functions (safety risk management and safety assurance) and integrating safety-relevant information across all relevant stakeholders. This allows a more effective use of safety information and provides an improved overview on, and prediction of, safety risks and ultimately improves the safety level of airports and their stakeholders. Furthermore, the methodology employed in this thesis is flexible and could be applied to all aspects of aviation SMS and system analysis.Open Acces