Autonomy Software: V&V Challenges and Characteristics

The successful operation of unmanned air vehicles requires software with a high degree of autonomy. Only if high level functions can be carried out without human control and intervention, complex missions in a changing and potentially unknown environment can be carried out successfully. Autonomy software is highly mission and safety critical: failures, caused by flaws in the software cannot only jeopardize the mission, but could also endanger human life (e.g., a crash of an UAV in a densely populated area). Due to its large size, high complexity, and use of specialized algorithms (planner, constraint-solver, etc.), autonomy software poses specific challenges for its verification, validation, and certification. -- - we have carried out a survey among researchers aid scientists at NASA to study these issues. In this paper, we will present major results of this study, discussing the broad spectrum. of notions and characteristics of autonomy software and its challenges for design and development. A main focus of this survey was to evaluate verification and validation (V&V) issues and challenges, compared to the development of "traditional" safety-critical software. We will discuss important issues in V&V of autonomous software and advanced V&V tools which can help to mitigate software risks. Results of this survey will help to identify and understand safety concerns in autonomy software and will lead to improved strategies for mitigation of these risks

Impact of maintainability defects on code inspections

Software inspections are effective ways to detect defects early in the development process. In this paper, we analyze the impact of certain defect types on the effectiveness of code inspection. We conducted an experiment in an academic environment with 88 subjects to empirically investigate the effect of two maintainability defects, i.e., indentation and naming conventions, on the number of functional defects found, the effectiveness of functional defect detections, and the number of false positives reported during individual code inspections. Results show that in cases where both naming conventions and indentation defects exist, the participants found minimum number of defects and reported the highest number of false positives, as compared to the cases where either indentation or naming defects exist. Among maintainability defects, indentation seems to significantly impact the number of functional defects found by the inspector, while the presence of naming conventions defects seems to have no significant impact on the number of functional defects detected. The presence of maintainability defects significantly impacts the number of false positives reported. On the effectiveness of individual code inspectors we observed no significant impact originated from the presence of indentation or naming convention defects. © 2010 ACM

Development of Advanced Verification and Validation Procedures and Tools for the Certification of Learning Systems in Aerospace Applications

Adaptive control technologies that incorporate learning algorithms have been proposed to enable automatic flight control and vehicle recovery, autonomous flight, and to maintain vehicle performance in the face of unknown, changing, or poorly defined operating environments. In order for adaptive control systems to be used in safety-critical aerospace applications, they must be proven to be highly safe and reliable. Rigorous methods for adaptive software verification and validation must be developed to ensure that control system software failures will not occur. Of central importance in this regard is the need to establish reliable methods that guarantee convergent learning, rapid convergence (learning) rate, and algorithm stability. This paper presents the major problems of adaptive control systems that use learning to improve performance. The paper then presents the major procedures and tools presently developed or currently being developed to enable the verification, validation, and ultimate certification of these adaptive control systems. These technologies include the application of automated program analysis methods, techniques to improve the learning process, analytical methods to verify stability, methods to automatically synthesize code, simulation and test methods, and tools to provide on-line software assurance

Implementasi Cross Site Scripting Vulnerability Assessment Tools Berdasarkan OWASP Code Review

Serangan cross site scripting (XSS) merupakan salah satu kerentanan yang paling sering ditemukan dalam aplikasi web. Sayangnya, tidak semua pengembang aplikasi dan tim security fasih terhadap kerentanan web secara menyeluruh (Khan et al., 2017). OWASP Code Review merupakan dokumen tertulis yang menjelaskan mengenai kaidah, aturan, dan standar yang baik dalam analisis kode aplikasi web. Selain itu, proses vulnerability assessment juga dapat membantu penemuan kerentanan dengan waktu yang lebih efisien. Penelitian ini akan membangun suatu sistem aplikasi yang dapat melakukan proses vulnerability assessment berdasarkan kaidah OWASP Code Review. Dalam perancangannya, didapatkan tujuh pola ekspresi regular yang dapat membantu mengidentifikasi jenis pelanggaran yang dilakukan dalam potongan kode program dan dua pola ekspresi regular utama untuk menemukan kerentanan. Selain itu, dirancang pula lima algoritme pendukung guna memahami bagaimana sistem akan diimplementasikan. Sistem ini diimplementasikan dalam kerangka kerja Django dan telah diuji berdasarkan validitas temuan, penggunaan cpu, dan response time. Berdasarkan hasil pengujian yang dilakukan, sistem yang dibangun terbukti dua kali lebih baik dalam penemuan kerentanan cross site scripting

What makes a code review trustworthy?

Code review is an important step during the process of certifying safetycritical software because only code that passes review can be implemented. Reviews are performed by review boards composed of highly skilled and experienced computer scientists, engineers and analysts who generally rely upon a checklist of properties ranging from high-level requirements to minute language details. While many checklists and coding standards exist, the actual decision of which properties are most important is generally based on the experience of the person in charge. This paper addresses the questions: How can code review ensure certification of trustworthy code? and Is code review trustworthy? We surveyed technical leaders at NASA and the Aerospace industry to find out which properties are most important during the code review. To make analyze easier, the most common properties have been classified along different ”views”, ranging from a standards-oriented view (defined as the properties needed to satisfy a specific standard) to a tool-oriented view. In this paper, we present this classification together with a summary of findings and feed-back from the survey. We also discuss how a more uniform view on properties of code review and tool capabilities can result in increased trust for safety-critical software.