Ensuring Safety of Machine Learning Components Using Operational Design Domain

The introduction of machine learning in the aviation domain is an ongoing process. This is also true for safety-critical domains, especially for the area of Urban Air Mobility. A significant growth in number of air taxis and an increasing level of autonomy is to be expected allowing for operating a large number of air taxis in complex urban environments. Due to the complexity of the tasks and the environment, key autonomy functions will be realized using machine learning, for example the camera-based detection of objects. However, the safety assurance for avionics systems using machine learning components is challenging. This work investigates safety and verification aspects of machine learning components. A camera-based detection of humans on the ground, e.g. to assess a potential landing area, serves as an example for an machine learning-based autonomy functio and was integrated into an Unmanned Aircraft. In the context of this exemplary machine learning component, the concept of Operational Design Domain as recently adapted European Aviation Safety Agency in the context of machine learning assurance is described along with other key concepts of machine learning assurance. Furthermore, runtime assurance is used to monitor conformance to the Operational Design Domain during flight. The presented flight test results indicate that monitoring the Operational Design Domain can support performance as well as the safety of the operation

Considerations of Artificial Intelligence Safety Engineering in Aerospace

Unmanned aircraft systems promise to be useful for a multitude of applications such as cargo transport and disaster recovery. The research on increased autonomous decision-making capabilities is therefore rapidly growing and advancing. However, the safe use, certification, and airspace integration for unmanned aircraft in a broad fashion is still unclear. Standards for development and verification of manned aircraft are either only partially applicable or resulting safety and verification efforts are unrealistic in practice due to the higher level of autonomy required by unmanned aircraft. Machine learning techniques are hard to interpret for a human and their outcome is strongly dependent on the training data. This work presents the current certification practices in unmanned aviation in the context of autonomy and artificial intelligence. Specifically, the recently introduced categories of unmanned aircraft systems and the specific operation risk assessment are described, which provide means for flight permission not solely focusing on the aircraft but also incorporating the target operation. Exemplary, we show how the specific operation risk assessment might be used as an enabler for hard-to-certify techniques by taking the operation into account during system design

Geofencing requirements for onboard safe operation monitoring

The new concept for operation of drones, published by EASA in 2015, enables new ways to influence and possibly reduce the necessary safety targets of certain system components without reducing the overall safety of the unmanned aircraft system (UAS). Based on the safety assessment, the specific category enables new aircraft system architectures and mission designs. In this context, this paper analyzes runtime monitoring as a strategy to contain the UAS in its operational volume. To assure predefined properties in flight and thus assure the safety of the operation in progress with a high robustness, a formal meth-odology for safe operation monitoring is utilized. With this approach, this work targets to link the concept of safe operation monitoring with the upcoming regulations regarding the specific category and the specific operation risk assessment (SORA). One particular aspect of this safe operation monitoring is geofencing, the capability to contain a UAS in a previously restricted area. In the regulatory framework of a specific operation, risk assessment is required and so is the containment of the UAS in its operational volume. The functional and safety requirements for geofencing regarding their impact on the underlying specific operation risk assessment are discussed. To facilitate this discussion, a taxonomy of geofencing characteristics is derived based on a literature survey. Consequently, the geofencing requirements are assessed regarding their robustness and applicability for certification purposes. As a result, by monitoring the integrity of the system at runtime using geofencing as an example, it is investigated if the requirements and thus costs of development and certification process for the remaining components can be reduced

Programma, De Optica, Quo Illustrissimo & Celissimo Comite ac Domino, Dn. Alberto Antonio, S. R. I. Quatuor-Viro, Comite Schwartzburgi & Hohnsteinii ... Scholæ Provincialis Nutritio, Consentiente, ad Panegyrin Autumnalem applaudentibus animis celebrandam, & opticas quasdam speculationes instituendas, Omnium Ordinum Eruditos unà cum honoratissimo Domino Rectore, invitat

PROGRAMMA, DE OPTICA, QUO ILLUSTRISSIMO & CELISSIMO COMITE AC DOMINO, DN. ALBERTO ANTONIO, S. R. I. QUATUOR-VIRO, COMITE SCHWARTZBURGI & HOHNSTEINII ... SCHOLÆ PROVINCIALIS NUTRITIO, CONSENTIENTE, AD PANEGYRIN AUTUMNALEM APPLAUDENTIBUS ANIMIS CELEBRANDAM, & OPTICAS QUASDAM SPECULATIONES INSTITUENDAS, OMNIUM ORDINUM ERUDITOS UNÀ CUM HONORATISSIMO DOMINO RECTORE, INVITAT Programma, De Optica, Quo Illustrissimo & Celissimo Comite ac Domino, Dn. Alberto Antonio, S. R. I. Quatuor-Viro, Comite Schwartzburgi & Hohnsteinii ... Scholæ Provincialis Nutritio, Consentiente, ad Panegyrin Autumnalem applaudentibus animis celebrandam, & opticas quasdam speculationes instituendas, Omnium Ordinum Eruditos unà cum honoratissimo Domino Rectore, invitat (1) Titelseite (1) Turpius emittit, quodcunque recepit, ocellus, ... (2

Machine Learning Verification and Safety for Unmanned Aircraft - A Literature Study

Machine learning (ML) has proven to be the tool of choice for achieving human-like or even super-human performance with automation on specific tasks. As a result, this data-driven approach is currently experiencing massive interest in all industry domains. This increased use also applies for the safety critical aviation domain. With no human pilot on board, the potential use cases of ML for unmanned aircraft are particularly promising. Even upcoming Urban Air Mobility (UAM) concepts are planning to remove the onboard pilot and instead use ML to support a remote pilot, possibly supervising a fleet of vehicles. However, the verification of ML algorithms is a challenging problem, since established safety standards and assurance methods are not applicable. Thus, this work comprises a literature study on the topic of ML verification and safety. This research paper uses a systematic approach to map and categorize the research and focus on specific subtopics that are of particular interest in the context of existing guidance documents

A Hierarchy of Monitoring Properties for Autonomous Systems

Monitoring capabilities play a central role in mitigating safety risks of current, and especially future autonomous aircraft systems. These future systems are likely to include complex components such as neural networks for environment perception, which pose a challenge for current verification approaches; they are considered as black-box components. To assure that these black-boxes comply with their specification, they must be monitored to detect violations during execution with respect to their input and output behaviors. Such behavioral properties often include more complex aspects such as temporal or spatial notions. The outputs can also be compared to data from other assured sensors or components of the aircraft, making monitoring an integral part of the system, which ideally has access to all available resources to assess the overall health of the operation. Current approaches using handwritten code for monitoring functions run the risk of not being able to keep up with these challenges. Therefore, in this paper, we present a hierarchy of monitoring properties that provides a perspective for overall health. We also present a categorization of monitoring properties and show how different monitoring specification languages can be used for formalization. These monitoring languages represent a higher abstraction of general-purpose code and are therefore more compact and easier for a user to write and read, and we can validate their implementations independently from the systems they reason about. They improve the maintainability of monitoring properties that is required to handle the increased complexity of future autonomous aircraft systems.This is a pre-print of the article Schirmer, Sebastian, Christoph Torens, Johann C. Dauer, Jan Baumeister, Bernd Finkbeiner, and Kristin Y. Rozier. "A Hierarchy of Monitoring Properties for Autonomous Systems." Copyright 2022 The Authors. Posted with permission

A Hierarchy of Monitoring Properties for Autonomous Systems

Monitoring capabilities play a central role in mitigating safety risks of current, but especially future autonomous aircraft systems. These future systems are likely to include complex components such as neural networks for environment perception, which pose a challenge for current verification approaches; they are considered as black-box components. To assure that these black-boxes comply to their specification, they are typically monitored to detect violations during execution in respect to their input and output behavior. Such behavioral properties often include more complex aspects such as temporal or spatial notions. Besides monitoring their behavior, the outputs can also be compared to data from other assured sensors or components of the aircraft, making monitoring an even more integral part of the system, which ideally has access to all available resources to assess the overall health of the operation. Current approaches using handwritten code for monitoring functions run the risk of not being able to keep up with these challenges. Therefore, in this paper, we present a hierarchy of monitoring properties that provides a perspective for overall health. We also present a categorization of monitoring properties and show how different monitoring specification languages can be used for formalization. These monitoring languages represent a higher abstraction of general-purpose code and are therefore more compact and easier for a user to write and read. They improve the maintainability of monitoring properties that is required to handle the increased complexity of future autonomous aircraft systems

Certification Aspects of Runtime Assurance for Urban Air Mobility

The transition towards autonomous operations for Urban Air Mobility introduces significant safety challenges, necessitating novel safety assurance strategies. One such strategy is runtime assurance, which ensures the safe behavior of a system during its actual operation. This can be implemented by using a safety monitor that detects unsafe behaviors and then activates a switch to a recovery function to return the system to a safe state. This paper investigates the certification aspects of runtime monitoring, a core component of runtime assurance. We analyze the regulatory framework of urban air mobility, and discuss implications of aviation software standards such as DO-178C and it supplements on runtime assurance. As a concrete example to discuss, Detect-and-Avoid is introduced and motivated from the requirements of the Minimum Operational Performance Standards. The use case is analyzed from a system and a software perspective. From a system perspective, the architecture is compared to the runtime assurance standard practice published by ASTM International. From a software perspective, we assess the stream-based specification language RTLola against the development assurance objectives in the de-facto software development standard DO-178C. As an example, we highlight the role of traceability between the different levels of software requirements. The goal of this research is to illustrate the use of runtime monitoring in the context of certification for Urban Air Mobility applications to improve operational safety and enable increasing levels of automation