Identification of test cases for Automated Driving Systems using Bayesian optimization

With advancements in technology, the automotive industry is experiencing a paradigm shift from assisted driving to highly automated driving. However, autonomous driving systems are highly safety critical in nature and need to be thoroughly tested for a diverse set of conditions before being commercially deployed. Due to the huge complexities involved with Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS), traditional software testing methods have well-known limitations. They also fail to cover the infinite number of adverse conditions that can occur due to a slight change in the interactions between the environment and the system. Hence, it is important to identify test conditions that push the vehicle under test to breach its safe boundaries. Hazard Based Testing (HBT) methods, inspired by Systems-Theoretic Process Analysis (STPA), identify such parameterized test conditions that can lead to system failure. However, these techniques fall short of discovering the exact parameter values that lead to the failure condition. The presented paper proposes a test case identification technique using Bayesian Optimization. For a given test scenario, the proposed method learns parameter values by observing the system's output. The identified values create test cases that drive the system to violate its safe boundaries. STPA inspired outputs (parameters and pass/fail criteria) are used as inputs to the Bayesian Optimization model. The proposed method was applied to an SAE Level-4 Low Speed Automated Driving (LSAD) system which was modelled in a driving simulator

Challenges in Future Mathematical Modelling of Hierarchical Functional Safety Control Structures within STAMP Safety Model

In the STAMP model, based on control theory, the control relationships between various system elements enforced by the closed Control Loops (CLs) are logical and functional. A literature survey emphasized the fact that for the moment STAMP and its main tools STPA and CAST are not associated with any numerical tools. The main rationale of our work is to understand whether STAMP matches to be a quantitative model. Furthermore, in a case that we find that numerical tools can be used in STAMP, we intend to bridge the gap between the logical-functional approach in STAMP and any of the suitable quantitative approaches applied in Engineering Control Theory (ECT). As a first step, a literature comparison was performed between the basic control parameters existing explicitly at the moment in the STAMP model, and those well known in the literature of ECT. The results reveal that there are many similar terms, especially related to conceptual and general definitions. However, we have observed that there are also basic quantitative parameters from ECT which are not yet referred to in STAMP as quantitative safety evaluation parameters. Another main finding is an inherent difference in various ECT related parameters and the CLs at the various hierarchical levels. ECT was originally developed to deal with physical systems. Thus, any machine related internal control loops within the lower-physical level of a Sociotechnical System (STS) can be directly addressed with quantitative methods from ECT. However, most of the human-machine interactions in the lower levels and the human and societal controls in the higher levels are at the moment not suitable for those methods. We assume these ECT parameters may have an important role in designing and examining systems safety and hence we suggest, should be integrated into STAMP model, in purpose to be able to enhance systems safety

Usage and Scaling of an Open-Source Spiking Multi-Area Model of Monkey Cortex

We are entering an age of `big' computational neuroscience, in which neural network models are increasing in size and in numbers of underlying data sets. Consolidating the zoo of models into large-scale models simultaneously consistent with a wide range of data is only possible through the effort of large teams, which can be spread across multiple research institutions. To ensure that computational neuroscientists can build on each other's work, it is important to make models publicly available as well-documented code. This chapter describes such an open-source model, which relates the connectivity structure of all vision-related cortical areas of the macaque monkey with their resting-state dynamics. We give a brief overview of how to use the executable model specification, which employs NEST as simulation engine, and show its runtime scaling. The solutions found serve as an example for organizing the workflow of future models from the raw experimental data to the visualization of the results, expose the challenges, and give guidance for the construction of ICT infrastructure for neuroscience

Corporate innovation systems and the effect of continuity, competence, and cooperation on innovation performance

Innovations have always been an essential factor for the long-term success of corporations. This is all the more true at times like the present, which is becoming increasingly dynamic and fast due to such effects as digitalization and globalization. However, as important as innovations are for the success of corporations, their systematic development is just as challenging. This fact can be demonstrated not least by numerous practical examples in which formerly successful corporations were unable to react appropriately to changing market and competitive conditions and consequently had to give up their market position. The challenges in the development of innovations can be traced back to different organizational conditions, which are necessary for the efficient exploitation of existing products on the one hand and the exploration of new innovations on the other. The scientific literature recommends, among other things, the separation of exploration and exploitation into different organizational units to meet the challenges mentioned above. In addition to the operational business units, which are usually responsible for the exploitation of existing products, it is advisable to establish innovation units, such as corporate incubators or corporate venture capital units, and to entrust them with the exploration of innovations. For a detailed examination of the current state of research on corporate incubators and corporate venture capital, two systematic literature analyses were carried out within the scope of this thesis. As a result, it was discovered that further research is needed, particularly concerning the organizational integration of such innovation units into the overall organization and the associated conflicts of objectives. To make an initial contribution to closing the research gap mentioned above, a further study of this work is devoted to the organizational integration of different innovation programs in an established corporation. This study differs from previous studies in that it takes an overarching perspective and considers the entire organization, including the innovation units, as a holistic innovation system. Such a corporate innovation system consists of at least three different types of innovation units in addition to the operational business units: exploration-oriented innovation units for the generation of disruptive innovations, exploitation-oriented innovation units for the further development of existing products and transformation-oriented innovation units for the transformation of the corporate culture. Such a system can ensure the systematic and sustainable generation of innovations, especially in the interaction of the various innovation units. In addition to the basic establishment of the innovation units mentioned above, however, appropriate organizational framework conditions are required to ensure that innovations can be developed successfully. The fourth study in this thesis is dedicated to the question of how continuity, competence and cooperation affect the innovation performance of corporations. It could be analyzed that the continuous implementation of innovation activities has the greatest positive effect on the innovation performance of enterprises. While cooperation, in combination with continuity, has a short- to medium-term impact on innovation performance, competence and continuity have a long-term effect on innovation performance. Cooperation and competence are complementary concepts in that cooperation should be used for short-term innovation activities, while competence should be used for the long-term sustainable development of innovations within the enterprise. As a result, this work addresses existing research gaps with regard to the integration of innovation units and the organizational structures of corporations and provides valuable insights and approaches for further research. For this purpose, it was necessary to link findings from the field of innovation management and corporate venturing with concepts of organizational theory. Through this connection, we have succeeded in gaining new scientific insights that previously could not be gained independently within the individual research streams. We are convinced that our findings on Corporate Innovation Systems and the effects of continuity, competence and cooperation on innovation performance have made an important scientific contribution. That is all the more true at a time when successful innovation is becoming increasingly important for corporations and a growing number of newly emerging innovation units can be observed in practice.Innovationen stellen einen wesentlichen Faktor für den langfristigen Erfolg von Unternehmen dar. Dies gilt umso mehr in einer Zeit wie der heutigen, welche durch Effekte wie die Digitalisierung und Globalisierung zunehmend an Dynamik und Schnelligkeit gewinnt. So bedeutsam Innovationen jedoch für den Erfolg von Unternehmen sind, so herausfordernd stellt sich deren systematische Entwicklung dar. Dies lässt sich nicht zuletzt an zahlreichen Praxisbeispielen belegen, in welchen ehemals erfolgreiche Unternehmen nicht in der Lage waren angemessen auf veränderte Markt- und Wettbewerbsbedingungen zu reagieren und in der Folge ihre Marktposition aufgeben mussten. Die Herausforderungen bei der Entwicklung von Innovationen lassen sich dabei insbesondere auf unterschiedliche organisatorische Voraussetzungen zurückführen, welche einerseits für die effiziente Exploitation bestehender Produkte und andererseits für die Exploration neuer Innovationen benötigt werden. Zur Begegnung der Herausforderungen wird in der wissenschaftlichen Literatur unter anderem die Trennung von Exploration und Exploitation in verschiedene Organisationeinheiten empfohlen. Neben den operativen Geschäftseinheiten, welche für die Exploitation bestehender Produkte verantwortlich sind, empfiehlt es sich daher Innovationseinheiten wie beispielsweise Corporate Incubators oder Corporate Venture Capital Einheiten zu etablieren und diese mit der Exploration neuer Innovation zu betrauen. Zur Untersuchung des aktuellen Forschungsstands zu Corporate Incubators und Corporate Venture Capital wurden im Rahmen dieser Arbeit unter anderem zwei systematische Literaturanalysen durchgeführt. Im Ergebnis konnte hierdurch aufgedeckt werden, dass es insbesondere hinsichtlich der organisatorischen Einbindung solcher Innovationseinheiten in die Gesamtorganisation und damit verbundener Zielkonflikte noch weiterer Forschung bedarf. Um einen ersten Beitrag zur Schließung der angeführten Forschungslücke zu leisten, widmet sich eine weitere Studie dieser Arbeit der organisatorischen Einbindung unterschiedlicher Innovationsprogramme in ein etabliertes Unternehmen. Dabei differenziert sich diese Studie von vorangegangenen Arbeiten, indem sie eine übergreifende Perspektive einnimmt und die Gesamtorganisation samt der Innovationseinheiten als ein holistisches Innovationssystem (Corporate Innovation System) betrachtet. Ein solches Corporate Innovation System besteht dabei neben den operativen Geschäftseinheiten aus mindestens drei verschiedenen Typen von Innovationseinheiten: Exploration-orientierten Innovationseinheiten für die Generierung disruptiver Innovationen, Exploitation-orientierten Innovationseinheiten für die Weiterentwicklung bestehender Produkten sowie Transformation-orientierte Innovationseinheiten für die Transformation der Unternehmenskultur. Insbesondere im Zusammenspiel der verschiedenen Innovationseinheiten kann dabei ein solches System die systematische und nachhaltige Generierung von Innovationen gewährleisten. Neben der grundsätzlichen Etablierung der angeführten Innovationseinheiten bedarf es jedoch zusätzlich entsprechender organisatorischer Rahmenbedingungen damit Innovationen erfolgreich entwickelt werden können. Hierzu widmet sich die vierte Studie dieser Arbeit der Frage, wie sich Kontinuität, Kompetenz und Kooperation auf die Innovationsleistung von Unternehmen auswirken. Hierbei konnte analysiert werden, dass die kontinuierliche Durchführung von Innovationsaktivitäten die größte positive Auswirkung auf die Innovationsleistung von Unternehmen hat. Während sich Kooperationen im Zusammenspiel mit Kontinuität insbesondere kurz- bis mittelfristig auf die Innovationsleistung auswirken, wirken Kompetenz und Kontinuität langfristig auf die Innovationsleistung. Kooperationen und Kompetenz stellen hierbei sich ergänzende Konzepte dar, indem Kooperation für kurzfristige Innovationsmaßnahmen eingesetzt werden sollten, während die Kompetenz im eigenen Unternehmen langfristig für die nachhaltige Entwicklung von Innovationen zum Einsatz kommen sollte. Im Ergebnis nimmt sich diese Arbeit bestehender Forschungslücken hinsichtlich der Einbindung von Innovationseinheiten und die Organisationsstrukturen von Unternehmen an und liefert hierzu wertvolle Erkenntnisse sowie Ansätze für weitere Forschungsarbeiten. Wir sind überzeugt, dass unsere Erkenntnisse über Corporate Innovation Systems und die Auswirkungen von Kontinuität, Kompetenz und Kooperation auf die Innovationsleistung einen wichtigen wissenschaftlichen Beitrag geleistet haben. Dies gilt umso mehr in einer Zeit, in der erfolgreiche Innovation für Unternehmen immer wichtiger wird und in der eine wachsende Zahl neu entstehender Innovationseinheiten in der Praxis zu beobachten ist

Considerations in Assuring Safety of Increasingly Autonomous Systems

Recent technological advances have accelerated the development and application of increasingly autonomous (IA) systems in civil and military aviation. IA systems can provide automation of complex mission tasks-ranging across reduced crew operations, air-traffic management, and unmanned, autonomous aircraft-with most applications calling for collaboration and teaming among humans and IA agents. IA systems are expected to provide benefits in terms of safety, reliability, efficiency, affordability, and previously unattainable mission capability. There is also a potential for improving safety by removal of human errors. There are, however, several challenges in the safety assurance of these systems due to the highly adaptive and non-deterministic behavior of these systems, and vulnerabilities due to potential divergence of airplane state awareness between the IA system and humans. These systems must deal with external sensors and actuators, and they must respond in time commensurate with the activities of the system in its environment. One of the main challenges is that safety assurance, currently relying upon authority transfer from an autonomous function to a human to mitigate safety concerns, will need to address their mitigation by automation in a collaborative dynamic context. These challenges have a fundamental, multidimensional impact on the safety assurance methods, system architecture, and V&V capabilities to be employed. The goal of this report is to identify relevant issues to be addressed in these areas, the potential gaps in the current safety assurance techniques, and critical questions that would need to be answered to assure safety of IA systems. We focus on a scenario of reduced crew operation when an IA system is employed which reduces, changes or eliminates a human's role in transition from two-pilot operations

An investigation into hazard-centric analysis of complex autonomous systems

This thesis proposes a hypothesis that a conventional, and essentially manual, HAZOP process can be improved with information obtained with model-based dynamic simulation, using a Monte Carlo approach, to update a Bayesian Belief model representing the expected relations between cause and effects – and thereby produce an enhanced HAZOP. The work considers how the expertise of a hazard and operability study team might be augmented with access to behavioural models, simulations and belief inference models. This incorporates models of dynamically complex system behaviour, considering where these might contribute to the expertise of a hazard and operability study team, and how these might bolster trust in the portrayal of system behaviour. With a questionnaire containing behavioural outputs from a representative systems model, responses were collected from a group with relevant domain expertise. From this it is argued that the quality of analysis is dependent upon the experience and expertise of the participants but this might be artificially augmented using probabilistic data derived from a system dynamics model. Consequently, Monte Carlo simulations of an improved exemplar system dynamics model are used to condition a behavioural inference model and also to generate measures of emergence associated with the deviation parameter used in the study. A Bayesian approach towards probability is adopted where particular events and combinations of circumstances are effectively unique or hypothetical, and perhaps irreproducible in practice. Therefore, it is shown that a Bayesian model, representing beliefs expressed in a hazard and operability study, conditioned by the likely occurrence of flaw events causing specific deviant behaviour from evidence observed in the system dynamical behaviour, may combine intuitive estimates based upon experience and expertise, with quantitative statistical information representing plausible evidence of safety constraint violation. A further behavioural measure identifies potential emergent behaviour by way of a Lyapunov Exponent. Together these improvements enhance the awareness of potential hazard cases

Conceptual Systems Security Analysis Aerial Refueling Case Study

In today’s highly interconnected and technology reliant environment, systems security is rapidly growing in importance to complex systems such as automobiles, airplanes, and defense-oriented weapon systems. While systems security analysis approaches are critical to improving the security of these advanced cyber-physical systems-of-systems, such approaches are often poorly understood and applied in ad hoc fashion. To address these gaps, first a study of key architectural analysis concepts and definitions is provided with an assessment of their applicability towards complex cyber-physical systems. From this initial work, a definition of cybersecurity architectural analysis for cyber-physical systems is proposed. Next, the System Theory Theoretic Process Analysis approach for Security (STPA Sec) is tailored and presented in three phases which support the development of conceptual-level security requirements, applicable design-level criteria, and architectural-level security specifications. This work uniquely presents a detailed case study of a conceptual-level systems security analysis of a notional aerial refueling system based on the tailored STPA-Sec approach. This work is critically important for advancing the science of systems security engineering by providing a standardized approach for understanding security, safety, and resiliency requirements in complex systems with traceability and testability

Using a systems approach to analyze the operational safety of dams

Dam systems are arrangements of interacting components that store and convey water for beneficial purposes. Dam failures are associated with extreme consequences to human life, the environment and the economy. Existing techniques for dam safety analysis tend to focus on verifying system performance at the edge of the design envelope. In analyzing the events which occur within the design envelope, linear chain-of-events models are often used to analyze the potential outcomes for the system. These chain-of-events models require that combinations of conditions are identified at the outset of the analysis, which can be very cumbersome given the number of physically possible combinations. Additional complications arising from feedback behaviour and time are not easily overcome using existing tools. Recent work in the industry has begun to focus on systems approaches to the problem, especially stochastic simulation. Given current computational abilities, stochastic simulation may not be capable of analyzing combinations of events that have a low combined probability but potentially extreme consequences. This research focuses on developing and implementing a methodology that dynamically characterizes combinations of component operating states and their potential impacts on dam safety. Automated generation of scenarios is achieved through the use of a component operating states database that defines all possible combinations of component states (scenarios) using combinatorics. A Deterministic Monte Carlo simulation framework systematically characterizes each scenario through a number of iterations that vary adverse operating state timing, impacts and inflows. Component interactions and feedbacks are represented within the system dynamics simulation model. Simulation outcomes provide useful indicators for dam operators including conditional failure rates, times to failure, failure inflow thresholds, and reservoir level exceedance frequencies. Dynamic system response can be assessed directly from the simulation outcomes. The scenario results may be useful to dam owners in emergency decision-making to inform response timelines and to justify the allocation of resources. Results may also help inform the development of improved operating strategies or upgrade alternatives that can reduce the impacts of these extreme events. This work offers a significant improvement in the ability to systematically characterize the potential combinations of events and their consequences