Development of Advanced Verification and Validation Procedures and Tools for the Certification of Learning Systems in Aerospace Applications

Adaptive control technologies that incorporate learning algorithms have been proposed to enable automatic flight control and vehicle recovery, autonomous flight, and to maintain vehicle performance in the face of unknown, changing, or poorly defined operating environments. In order for adaptive control systems to be used in safety-critical aerospace applications, they must be proven to be highly safe and reliable. Rigorous methods for adaptive software verification and validation must be developed to ensure that control system software failures will not occur. Of central importance in this regard is the need to establish reliable methods that guarantee convergent learning, rapid convergence (learning) rate, and algorithm stability. This paper presents the major problems of adaptive control systems that use learning to improve performance. The paper then presents the major procedures and tools presently developed or currently being developed to enable the verification, validation, and ultimate certification of these adaptive control systems. These technologies include the application of automated program analysis methods, techniques to improve the learning process, analytical methods to verify stability, methods to automatically synthesize code, simulation and test methods, and tools to provide on-line software assurance

A Review of Verification and Validation for Space Autonomous Systems

Purpose of Review: The deployment of hardware (e.g., robots, satellites, etc.) to space is a costly and complex endeavor. It is of extreme importance that on-board systems are verified and validated through a variety of verification and validation techniques, especially in the case of autonomous systems. In this paper, we discuss a number of approaches from the literature that are relevant or directly applied to the verification and validation of systems in space, with an emphasis on autonomy. Recent Findings: Despite advances in individual verification and validation techniques, there is still a lack of approaches that aim to combine different forms of verification in order to obtain system-wide verification of modular autonomous systems. Summary: This systematic review of the literature includes the current advances in the latest approaches using formal methods for static verification (model checking and theorem proving) and runtime verification, the progress achieved so far in the verification of machine learning, an overview of the landscape in software testing, and the importance of performing compositional verification in modular systems. In particular, we focus on reporting the use of these techniques for the verification and validation of systems in space with an emphasis on autonomy, as well as more general techniques (such as in the aeronautical domain) that have been shown to have potential value in the verification and validation of autonomous systems in space

A Review of Verification and Validation for Space Autonomous Systems

From Springer Nature via Jisc Publications RouterHistory: registration 2021-05-13, accepted 2021-05-13, online 2021-06-18, pub-electronic 2021-06-18, pub-print 2021-09Publication status: PublishedFunder: Engineering and Physical Sciences Research Council; doi: https://doi.org/10.13039/501100000266; Grant(s): EP/R026092/1Abstract: Purpose of Review: The deployment of hardware (e.g., robots, satellites, etc.) to space is a costly and complex endeavor. It is of extreme importance that on-board systems are verified and validated through a variety of verification and validation techniques, especially in the case of autonomous systems. In this paper, we discuss a number of approaches from the literature that are relevant or directly applied to the verification and validation of systems in space, with an emphasis on autonomy. Recent Findings: Despite advances in individual verification and validation techniques, there is still a lack of approaches that aim to combine different forms of verification in order to obtain system-wide verification of modular autonomous systems. Summary: This systematic review of the literature includes the current advances in the latest approaches using formal methods for static verification (model checking and theorem proving) and runtime verification, the progress achieved so far in the verification of machine learning, an overview of the landscape in software testing, and the importance of performing compositional verification in modular systems. In particular, we focus on reporting the use of these techniques for the verification and validation of systems in space with an emphasis on autonomy, as well as more general techniques (such as in the aeronautical domain) that have been shown to have potential value in the verification and validation of autonomous systems in space

Establishing a Performance Testing Approach for E-Learning Applications

Most of the E-Learning applications perform poorly in motivating employees to learn. To solve this problem, we need to examine what workplace e-learning requires and how workplace e-learning systems should be developed in line with those requirements. We investigated the problem by identifying the fundamental elements of the workplace learning environment including the learner, organization, learning content and social context, and their relationships. We found that workplace e-learning should align individual and organizational learning needs, connect learning and work performance, and support social interaction among individuals. To achieve this, a performance testing approach is proposed. Key performance indicators are utilized to clarify organizational goals, make sense of work context and requests on work performance, and accordingly help employees set up rational learning objectives and enhance their learning process. Using this approach, prototype system has been developed and a set of experiments have been conducted to demonstrate the effectiveness of the approach. This paper also presents the use of software verification, validation and testing technique, traditionally used in the software development, in the design and implementation of E-Learning products. We examine the ways one can apply testing techniques in E-Learning life cycle. This includes the strategy adoption for the selection of testing technique along with tool acquisition and measurement. The objective is to develop a collaborative approach involving software testing and educational methodology

Minimal structures for program analysis and verification

The reliability of software systems is a key societal concern. Because software should provide versatile functionalities in a myriad of contexts, the conception, design, and validation of programs remains a complex task.Techniques for (automated) software verification are very important. Nowadays, it is widely accepted that formal methods can play a decisive role towards reliable software. Formal methods refer to a broad set of mathematical techniques that allow us to precisely describe software and its intended properties. Relying on mathematical foundations, these techniques provide strong guarantees for correctness.The practical adoption of formal methods, however, still lags behind. There are two key challenges: (i) an increased learning curve for developers, and (ii) the associated analysis techniques come with their own implementation complexity. There is both practical and theoretical interest in rigorously studying simpler, or even minimal, formulations of the mathematical structures that underpin formal methods.In this thesis, we focus on two such structures: typestates and session types. They both specify software properties that vary over time (i.e., temporal properties). We provide evidence that simpler formulations of these structures can lead to both practical and theoretical benefits. On the practical side, our formulations enable considerable usability and performance improvements and streamlined embedding into mainstream programming languages. On the theoretical side, we establish a precise relationship between our formulations and the original ones, fully justifying the benefits of adopting simplerformulations

Validation and Verification of Aircraft Control Software for Control Improvement

Validation and Verification are important processes used to ensure software safety and reliability. The Cooper-Harper Aircraft Handling Qualities Rating is one of the techniques developed and used by NASA researchers to verify and validate control systems for aircrafts. Using the Validation and Verification result of controller software to improve controller\u27s performance will be one of the main objectives of this process. Real user feedback will be used to tune PI controller in order for it to perform better. The Cooper-Harper Aircraft Handling Qualities Rating can be used to justify the performance of the improved system