Identifying how automation can lose its intended benefit along the development process : a research plan

Doctoral Consortium Presentation © The Authors 2009Automation is usually considered to improve performance in virtually any domain. However it can fail to deliver the target benefit as intended by those managers and designers advocating the introduction of the tool. In safety critical domains this problem is of significance not only because the unexpected effects of automation might prevent its widespread usage but also because they might turn out to be a contributor to incident and accidents. Research on failures of automation to deliver the intended benefit has focused mainly on human automation interaction. This paper presents a PhD research plan that aims at characterizing decisions for those involved in development process of automation for safety critical domains, taken under productive pressure, to identify where and when the initial intention the automation is supposed to deliver can be lost along the development process. We tentatively call such decisions as drift and the final objective is to develop principles that will allow to identify and compensate for possible sources of drift in the development of new automation. The research is based on case studies and is currently entering Year 2

Pragmatic science? Reflections on the academic - industry interactions in a European aviation research programme

In this article we explore the types of science produced and disseminated in human factors research in the cognitive domain. We reflect on the methods and techniques used in the European, Horizon 2020 Future Sky Safety Project: Human Performance Envelope (P6). This project has spanned multiple research paradigms successfully, and engaged academia and research organizations throughout. We discuss the challenges of conducting practically focused work that can also be brought to publication in peer-reviewed journals. Finally, we frame our research contributions within this project using a model of practitioner engagement

Safety sans frontières: an international safety culture model

The management of safety culture in international and culturally diverse organisations is a concern for many high-risk industries. Yet, research has primarily developed models of safety culture within Western countries, and there is a need to extend investigations of safety culture to global environments. We examined i) whether safety culture can be reliably measured within a single industry operating across different cultural environments, and ii) if there is an association between safety culture and national culture. The psychometric properties of a safety culture model developed for the air traffic management industry (ATM) were examined in 17 European countries from four culturally distinct regions of Europe (North, East, South, West). Participants were ATM operational staff (n = 5176) and management staff (n = 1230). Through employing multi-group confirmatory factor analysis, good psychometric properties of the model were established. This demonstrates, for the first time, that when safety culture models are tailored to a specific industry, they can operate consistently across national boundaries and occupational groups. Additionally, safety culture scores at both regional and national levels were associated with country-level data on Hofstede’s five national culture dimensions (collectivism, power distance, uncertainty avoidance, masculinity, and uncertainty avoidance). MANOVAs indicated safety culture to be most positive in Northern Europe, less so in Western and Eastern Europe, and least positive in Southern Europe. This indicates that national cultural traits may influence the development of organisational safety culture, with significant implications for safety culture theory and practice

The relationship between national culture and safety culture: implications for international safety culture assessments

In this article we examine the relationship between safety culture and national culture, and the implications of this relationship for international safety culture assessments. Focussing on Hofstede’s uncertainty avoidance (UA) index, a survey study of 13,616 Air Traffic Management (ATM) employees in 21 European countries found a negative association between safety culture and national norm data for uncertainty avoidance. This is theorized to reflect the influence of national tendencies for uncertainty avoidance upon attitudes and practices for managing safety (e.g., anxiety on risk; reliance on protocols; concerns over reporting incidents; openness to different perspectives). The relationship between uncertainty avoidance and safety culture is likely to have implications for international safety culture assessments. Specifically, benchmarking exercises will consistently indicate safety management within organizations in high UA countries to be poorer than low UA countries due to the influence of national culture upon safety practices, which may limit opportunities for identifying and sharing best practice. We propose the use of safety culture against international group norms (SIGN) scores to statistically adjust for the influence of uncertainty avoidance upon safety culture data, and to support the identification of safety practices effective and particular to low or high UA cultures

Safety culture and power: interactions between perceptions of safety culture, organisational hierarchy, and national culture

Practices that involve power dynamics are integral to maintaining organisational safety (e.g. speaking-up, challenging poor behaviour, admitting error, communicating on safety), and staff engagement in these is assumed to be shaped by perceptions of safety culture. These perceptions, in-turn, are associated with (1) positions within an organisational hierarchy (which makes power-related acts more or less threatening), and (2) societal values for power distance (e.g. challenging authority). With a sample of 13,573 of air traffic control staff (controllers, engineers, administrative, and management) from 21 national air traffic providers, we reconfirm the observation that managers perceive safety culture more positively than frontline staff (hypothesis 1), and that workers in countries with established values for hierarchy and power report safety culture as less positive than those from countries with low power distance (hypothesis 2). We then, for the first time, examine the interaction between these two factors, and establish that differences in safety culture perceptions between those higher in the hierarchy (management) and those lower in the hierarchy (air traffic controllers and administrative staff) are exacerbated by national contexts for large power distance (hypothesis 3). The study contributes to the literature by theorising the role of power in safety culture theory, and its influence upon safety culture perceptions. Moving forward, safety culture research and interventions may benefit from considering how power exists and manifests at the level of superior-subordinate dynamics

The application of human factors in wake vortex encounter flight simulations for the reduction of flight upset risk and startle response

The current top safety risk concern for commercial air travel in Europe is known as “Flight Upset”. This term, also known as “Loss of Control in Flight”, entails the flight crew suddenly finding themselves in an unexpected, complex, and even confusing situation that if not resolved quickly can lead to a major accident. Accidents such as AF447 and the two B737 Max accidents fall into this category. An undesirable aspect of such events is known as the “startle response”, wherein one or both flight crew, finding themselves in dire and dangerous conditions, may experience ‘startle’, which temporarily affects their cognitive functioning. This may only last half a minute, but its effect can have a severe impact on the survivability of such events. A Horizon 2020 research project called SAFEMODE, which aims to integrate Human Factors techniques into a unified framework for designers in aviation and maritime domains, is exploring the use of state-of-the-art flight simulation facilities to measure pilot performance in severe wake turbulence events, which can induce the startle effect. This is part of a broader use case within SAFEMODE to validate the design of a new Wake Vortex Air Traffic Alert for the Cruise phase of flight. A tactical short-term alert to the Flight Crew, ahead of the wake encounter, is seen as beneficial to reduce the startle effect and support the appropriate management of these conflicts. The envisaged risk-alerting logic relies on a ground-based predictor, connected to the Air Traffic Control system, displaying an alert to the En-route Air Traffic Controllers, who can then provide a cautionary advisory to the Flight Crew so they can take appropriate actions.The cockpit flight simulations involve type-rated flight crews in realistic and representative cruise flight conditions, using a Type VI Boeing 737-800 full flight motion-based simulator (also used for Upset Prevention and Recovery training programs). During the simulation runs, pilots are exposed to simulated wake vortex encounters, corresponding to a strong wake-induced upset (between 30 and 40 degrees of bank), with or without prior ATC wake caution, and varying the initial direction of roll between left and right to limit the simulation training effect.Human Factors measurements include workload, situation awareness, trust, acceptability-based user feedback, as well as psychophysiological measures such as eye-tracking and Electro-Dermal Activity (EDA). In particular, eye-tracking is expected to support the refined determination of the sequence of actions before and after detection, and the reaction of flight crews to the en-route ATC Wake alert.A cockpit flight simulation, via combining the analyses of psychophysiological measures, flight parameters, expert observations and subjective pilot feedback, enables evaluation of Flight Crews performance in preparing for, managing or avoiding wake encounter upsets with the new ATC wake alerts, showing the net safety benefits. Early results indicate that the simulations can indeed induce startle effect, and that repeated exposure enables flight crew to overcome it and manage the situation in a more measured and controlled fashion

Towards a safety learning culture for the shipping industry : a white paper

Within the framework of the EU-funded SAFEMODE project, a series of confidential, in-depth interviews of seafarers and investigators was carried out to ascertain the current status of Safety Culture in the shipping industry, and to recommend possible avenues for improvement. The interview script covered practices in incident and accident investigation and reporting, the Human Element, the factors that keep the ship safe, the role of the Safety Management System, Just Culture and Safety Learning. The seafarers’ and investigators’ interviews were complemented by small focus groups with unions, education and safety bodies. Participants were open and genuine in providing their opinions, as anonymity was preserved. The general consensus among interviewees was that seafarers are the ones who keep ships safe at sea, which is a good omen for Safety Culture in the shipping industry. The originally intended ‘destination’ for the shipping industry was to be Just Culture, but the interviews quickly revealed that Safety Learning, already evident in some parts of the industry, appeared a more pragmatic and attainable destination, one that could add safety improvements and shore up Safety Cultur

Advancing Safety in Organisations: Application via the Luton Safety Stack

Controlling and improving safety in organisations is achieved using a Safety Management System (SMS). Notwithstanding the variety of components considered in SMS standards, including human factors and safety culture, safety management systems are sometimes observed by those at the ‘sharp end’ as being bureaucratic, distinct from actual operations, and being too focused on the prevention of deviations from procedures, rather than on the effective support of safety in the real operational context. The soft parts of advancing safety in organisations, such as the multitude of interrelations and the informal aspects in an organisation that influence safety, are only considered to a limited extent in traditional safety management systems. The research in Future Sky Safety Project 5 (FSS P5) focused on improving these human-related, informal organisational aspects. Since every organisation is unique, in the operations it conducts, its history, and its organisational culture, there cannot be a one-size-fits-all standard for advancing safety in organisations. Rather, this needs to be based on the organisation at hand, leading to tailored solutions. This has been applied to a safety culture assessment and enhancement approach applied for six key organisations at London Luton Airport, and the approach has become known as the Luton Safety Stack. The six organisations decided to share the detailed results of their individual safety culture assessments. They formed a group that holds quarterly meetings, which always include a workshop element. From this approach, the organisations were stimulated to develop harmonised procedures for all ground-handling operators at Luton, and for each operation, creating a simple one-page procedure with diagrams, to keep it simple and safe. The Luton Safety Stack shows that when organisations share a place, such as at an airport, they need the opportunity to meet to discuss both potential safety threats, and opportunities to advance safety, because even through organisations are interdependent, safety issues in one organisation often have implications for others

Charting the edges of human performance

In the Horizon 2020 funded Future Sky Safety programme, the Human Performance Envelope project pushed airline pilots to the edges of their performance in real-time cockpit simulations, by increasing stress and workload, and decreasing situation awareness. The aim was to find out how such factors interact, and to detect the edges of human performance where some form of automation support should be employed to ensure safe continued flight. A battery of measures was used, from behavioural to physiological (e.g. heart rate, eye tracking and pupil dilation), to monitoring pilot performance in real time. Several measures – e.g. heart rate, heart rate variability, eye tracking, cognitive walkthrough, and Human Machine Interface (HMI) usability analysis – proved to be useful and relatively robust in detecting performance degradation, and determining where changes in information presentation are required to better support pilot performance in challenging situations. These results led to proposed changes in a prototype future cockpit human-machine interface, which were subsequently validated in a final simulation. The results also informed the development of a ‘Smart-Vest’ that can be worn by pilots to monitor a range of signals linked to performance

Application of a SOAM-based systemic method to conduct a comprehensive analysis of a maritime accident

The current trends in maritime accidents worldwide are often linked to environmental, economic, and human consequences, such as oil spills, insurance costs, or human injuries or fatalities. Despite the continuous improvement in safety measures, maritime accidents are still occurring, and this remains a major concern in our society. The main aim of this paper is to contribute to the current safety measures by identifying the significant human and organisational accident contributors, and therefore, reducing the current accident trends. With this aim, this paper proposes to apply a systemic method known as the Systemic Occurrence Analysis Methodology (SOAM) for the first time in the maritime domain. SOAM, which is a 'Swiss-cheese" based organisational technique for analysing incidents and accidents, was developed by EUROCONTROL for the aviation domain. SOAM methodology is fully applied to three maritime accident collisions to identify the major accident contributors, absent or failed barriers, human involvement, and contextual conditions