Vulnerabilities preservation using code mutation

Dissertação de mestrado integrado em Informatics EngineeringThe main goal of software security testing is to assess the security risks of an application so that programmers can eliminate all vulnerabilities, as early as possible, before they are exploited by attackers. There are several tools on the market that allow to perform these tests during the software development life cycle to ensure that there are no security flaws in the final product. However, like all tools, these can also have imperfections, one of them being unable to detect weaknesses in vulnerable software. The project of this dissertation aims to tackle this problem, so that it is possible to find and correct flaws in security tests in order to, consequently, increase the effectiveness of the tools that intend to certify the security of applications. For this, the solution studied in this document is to apply syntactic transformations in vulnerable code samples without interfering in the presence of the vulnerabilities that should later be detected. This process is based on: ) code refactoring techniques that allow improving the internal quality of the software; ) the mutation testing system used to evaluate the quality of software testing. To implement this idea, a tool called VSG was developed with the functionality of producing new code samples with security flaws. This document describes the whole development process, from the architecture to the implementation of the tool. In the end, there is an analysis with the results obtained when trying to detect the vulnerabilities present in the samples produced through the CxSAST application of the company Checkmarx, from which this dissertation emerged.O objetivo principal de testes de segurança de software consiste em avaliar os riscos de segurança de uma aplicação para que os programadores possam eliminar todas as vulnerabilidades o mais cedo possível, antes que sejam exploradas por atacantes. Existem várias ferramentas no mercado que permitem realizar estes testes durante o processo de desenvolvimento de software para garantir que não existam falhas de segurança no produto final. Porém, tal como todas as ferramentas, estas também podem apresentar imperfeições, sendo uma delas não conseguir detetar fraquezas em software vulnerável. O projeto desta dissertação pretende combater este problema, de modo a que seja possível encontrar e corrigir falhas nos testes de segurança para, consequentemente, aumentar a eficácia das ferramentas que pretendem certificar a segurança das aplicações. Para isto, a solução estudada neste documento passa por aplicar transformações sintáticas em amostras de código vulneráveis sem interferir na presença das vulnerabilidades que deverão, posteriormente, ser detetadas. Este processo baseia-se: ) nas técnicas de refatoração de código que permitem melhorar a qualidade interna do software; ) no sistema de testes de mutação usado para avaliar a qualidade de testes de software. Para implementar esta ideia, uma ferramenta chamada VSG foi desenvolvida com a funcionalidade de produzir novas amostras de código com falhas de segurança. Neste documento é descrito todo o processo de desenvolvimento, desde a arquitetura até à implementação da ferramenta. No final, existe uma análise com os resultados obtidos ao tentar detetar as vulnerabilidades presentes nas amostras produzidas através da aplicação CxSAST da empresa Checkmarx, da qual esta dissertação surgiu

A Test-Driven Approach to Developing Pointcut Descriptors in AspectJ

International audienceAspect-oriented programming (AOP) languages introduce new constructs that can lead to new types of faults, which must be targeted by testing techniques. In particular, AOP languages such as AspectJ use a pointcut descriptor (PCD) that provides a convenient way to declaratively specify a set of joinpoints in the program where the aspect should be woven. However, a major difficulty when testing that the PCD matches the intended set of joinpoints is the lack of precise specification for this set other than the PCD itself. In this paper, we propose a test-driven approach for the development and validation of the PCD. We developed a tool, AdviceTracer, which enriches the JUnit API with new types of assertions that can be used to specify the expected joinpoints. In order to validate our approach, we also developed a mutation tool that systematically injects faults into PCDs. Using these two tools, we perform experiments to validate that our approach can be applied for specifying expected joinpoints and for detecting faults in the PCD

Modeling and Testing of Aspect-Oriented Systems

Aspect-Oriented programming modularizes crosscutting concerns into Aspects, which are automatically weaved to the specified points of a program. Although Aspect-Oriented programming improves program maintainability and the encapsulation of crosscutting concerns, it also breaks some traditional programming rules due to the weaving mechanism. Therefore, a new software testing approach has to be developed to rigorously test Aspect-Oriented programs. In this thesis, we introduce the concept of Aspect modeling and Aspect testing and then proceed to our investigation of a model-based incremental approach for testing Aspect-Oriented programs. First, a state machine model in UML is created for each Aspect and each base class, which is the class to be weaved with the aspect. Then each individual aspect or base class is tested using the test cases generated from state machine models. A combined state machine model is established by weaving the aspect model into the base class model. Finally, we perform a test on the woven program using test cases generated from the combined state machine model. Because the number of scenarios for weaving aspects and base classes could be very large, it may require a huge number of test cases to effectively test the program. To speed up the process, we propose a prioritizing strategy for selecting test cases in order to find errors sooner since different test cases have different capacity for tracking errors. We demonstrate that the test cases generated from the state machine model have to satisfy the adequacy of the transition coverage, the round-trip coverage, and the state coverage in the state machine model. Furthermore, the prioritizing strategy is developed based on the number of changes brought by weaving of an aspect and its base classes. The test case including more changes will have a higher priority. The effectiveness of the investigated strategy is evaluated through the case study and the mutation testing. The result of case study shows that the model-based incremental approach integrated with prioritizing test case selection provides an effective tool for testing large-scale Aspect-Oriented systems.  M.S

Towards an Automation of the Mutation Analysis Dedicated to Model Transformation

International audienceA major benefit of Model Driven Engineering (MDE) relies on the automatic generation of artefacts from high-level models through intermediary levels using model transformations. In such a process, the input must be well-designed and the model transformations should be trustworthy. Due to the specificities of models and transformations, classical software test techniques have to be adapted. Among these techniques, mutation analysis has been ported and a set of mutation operators has been defined. However, mutation analysis currently requires a considerable manual work and suffers from the test data set improvement activity. This activity is seen by testers as a difficult and time-consuming job, and reduces the benefits of the mutation analysis. This paper addresses the test data set improvement activity. Model transformation traceability in conjunction with a model of mutation operators, and a dedicated algorithm allow to automatically or semi-automatically produce test models that detect new faults. The proposed approach is validated and illustrated in a case study written in Kermeta

Higher Order Mutation Testing

Mutation testing is a fault-based software testing technique that has been studied widely for over three decades. To date, work in this field has focused largely on first order mutants because it is believed that higher order mutation testing is too computationally expensive to be practical. This thesis argues that some higher order mutants are potentially better able to simulate real world faults and to reveal insights into programming bugs than the restricted class of first order mutants. This thesis proposes a higher order mutation testing paradigm which combines valuable higher order mutants and non-trivial first order mutants together for mutation testing. To overcome the exponential increase in the number of higher order mutants a search process that seeks fit mutants (both first and higher order) from the space of all possible mutants is proposed. A fault-based higher order mutant classification scheme is introduced. Based on different types of fault interactions, this approach classifies higher order mutants into four categories: expected, worsening, fault masking and fault shifting. A search-based approach is then proposed for locating subsuming and strongly subsuming higher order mutants. These mutants are a subset of fault mask and fault shift classes of higher order mutants that are more difficult to kill than their constituent first order mutants. Finally, a hybrid test data generation approach is introduced, which combines the dynamic symbolic execution and search based software testing approaches to generate strongly adequate test data to kill first and higher order mutants