Model-Based Security Testing

Security testing aims at validating software system requirements related to security properties like confidentiality, integrity, authentication, authorization, availability, and non-repudiation. Although security testing techniques are available for many years, there has been little approaches that allow for specification of test cases at a higher level of abstraction, for enabling guidance on test identification and specification as well as for automated test generation. Model-based security testing (MBST) is a relatively new field and especially dedicated to the systematic and efficient specification and documentation of security test objectives, security test cases and test suites, as well as to their automated or semi-automated generation. In particular, the combination of security modelling and test generation approaches is still a challenge in research and of high interest for industrial applications. MBST includes e.g. security functional testing, model-based fuzzing, risk- and threat-oriented testing, and the usage of security test patterns. This paper provides a survey on MBST techniques and the related models as well as samples of new methods and tools that are under development in the European ITEA2-project DIAMONDS.Comment: In Proceedings MBT 2012, arXiv:1202.582

Towards automatic generation of parameterized test cases from abstractions

Model-based tools for automatic test generation usually can handle systems of a rather limited size. Therefore, they cannot be applied directly to systems of real industrial size. Here, we propose an approach to test generation combining enumerative data abstraction, test generation methods and constraint solving. The approach allows applying enumerative test generation tools like TGV to large and infinite systems. Given such a system, abstractions allow to derive a finite abstract system suitable for automatic test generation with enumerative tools. Abstract test cases need to be parameterized with actual test data, in order to execute them. For data selection, we make use of constraint solving techniques. Test case execution will later be done by TTCN-

Model Based Security Testing for Autonomous Vehicles

The purpose of this dissertation is to introduce a novel approach to generate a security test suite to mitigate malicious attacks on an autonomous system. Our method uses model based testing (MBT) methods to model system behavior, attacks and mitigations as independent threads in an execution stream. The threads intersect at a rendezvous or attack point. We build a security test suite from a behavioral model, an attack type and a mitigation model using communicating extended finite state machine (CEFSM) models. We also define an applicability matrix to determine which attacks are possible with which states. Our method then builds a comprehensive test suite using edge-node coverage that allows for systematic testing of an autonomous vehicle

A formal framework for specification-based embedded real-time system engineering

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2008.Includes bibliographical references (v. 2, p. 517-545).The increasing size and complexity of modern software-intensive systems present novel challenges when engineering high-integrity artifacts within aggressive budgetary constraints. Among these challenges, ensuring confidence in the engineered system, through validation and verification activities, represents the high cost item on many projects. The expensive nature of engineering high-integrity systems using traditional approaches can be partly attributed to the lack of analysis facilities during the early phases of the lifecycle, causing the validation and verification activities to begin too late in the engineering lifecycle. Other challenges include the management of complexity, opportunities for reuse without compromising confidence, and the ability to trace system features across lifecycle phases. The use of models as a specification mechanism provides an approach to mitigate complexity through abstraction. Furthermore, if the specification approach has formal underpinnings, the use of models can be leveraged to automate engineering activities such as formal analysis and test case generation. The research presented in this thesis proposes an engineering framework which addresses the high cost of validation and verification activities through specification-based system engineering. More specifically, the framework provides an integrated approach to embedded real-time system engineering which incorporates specification, simulation, formal verification, and test-case generation. The framework aggregates the state-of-the-art in individual software engineering disciplines to provide an end-to-end approach to embedded real-time system engineering. The key aspects of the framework include: * A novel specification language, the Timed Abstract State Machine (TASM) language, which extends the theory of Abstract State Machines (ASM).(cont.) The TASM language is a literate formal specification language which can be applied and multiple levels of abstraction and which can express the three key aspects of embedded real-time systems - function, time, and resources. * Automated verification capabilities achieved through the integration of mature analysis engines, namely the UPPAAL tool suite and the SAT4J SAT solver. The verification capabilities provided by the framework include completeness and consistency verification, model checking, execution time analysis, and resource consumption analysis. * Bi-directional traceability of model features across levels of abstraction and lifecycle phases. Traceability is achieved syntactically through archetypical refinement types; each refinement type provides correctness criteria, which, if met, guarantee semantic integrity through the refinement. * Automated test case generation capabilities for unit testing, integration testing, and regression testing. Unit test cases are generated to achieve TASM specification coverage through the rule coverage criterion. Integration test case generation is achieved through the hierarchical composition of unit test cases. Regression test case generation is achieved by leveraging the bi-directional traceability of model features. The framework is implemented into an integrated tool suite, the TASM toolset, which incorporates the UPPAAL tool suite and the SAT4J SAT solver. The toolset and framework are evaluated through experimentation on three industrial case studies - an automated manufacturing system, a "drive-by-wire" system used at a major automotive manufacturer, and a scripting environment used on the International Space Station.by Martin Ouimet.Ph.D