Towards Debugging and Improving Adversarial Robustness Evaluations ​

Despite exhibiting unprecedented success in many application domains, machine‐learning models have been shown to be vulnerable to adversarial examples, i.e., maliciously perturbed inputs that are able to subvert their predictions at test time. Rigorous testing against such perturbations requires enumerating all possible outputs for all possible inputs, and despite impressive results in this field, these methods remain still difficult to scale to modern deep learning systems. For these reasons, empirical methods are often used. These adversarial perturbations are optimized via gradient descent, minimizing a loss function that aims to increase the probability of misleading the model’s predictions. To understand the sensitivity of the model to such attacks, and to counter the effects, machine-learning model designers craft worst-case adversarial perturbations and test them against the model they are evaluating. However, many of the proposed defenses have been shown to provide a false sense of security due to failures of the attacks, rather than actual improvements in the machine‐learning models’ robustness. They have been broken indeed under more rigorous evaluations. Although guidelines and best practices have been suggested to improve current adversarial robustness evaluations, the lack of automatic testing and debugging tools makes it difficult to apply these recommendations in a systematic and automated manner. To this end, we tackle three different challenges: (1) we investigate how adversarial robustness evaluations can be performed efficiently, by proposing a novel attack that can be used to find minimum-norm adversarial perturbations; (2) we propose a framework for debugging adversarial robustness evaluations, by defining metrics that reveal faulty evaluations as well as mitigations to patch the detected problems; and (3) we show how to employ a surrogate model for improving the success of transfer-based attacks, that are useful when gradient-based attacks are failing due to problems in the gradient information. To improve the quality of robustness evaluations, we propose a novel attack, referred to as Fast Minimum‐Norm (FMN) attack, which competes with state‐of‐the‐art attacks in terms of quality of the solution while outperforming them in terms of computational complexity and robustness to sub‐optimal configurations of the attack hyperparameters. These are all desirable characteristics of attacks used in robustness evaluations, as the aforementioned problems often arise from the use of sub‐optimal attack hyperparameters, including, e.g., the number of attack iterations, the step size, and the use of an inappropriate loss function. The correct refinement of these variables is often neglected, hence we designed a novel framework that helps debug the optimization process of adversarial examples, by means of quantitative indicators that unveil common problems and failures during the attack optimization process, e.g., in the configuration of the hyperparameters. Commonly accepted best practices suggest further validating the target model with alternative strategies, among which is the usage of a surrogate model to craft the adversarial examples to transfer to the model being evaluated is useful to check for gradient obfuscation. However, how to effectively create transferable adversarial examples is not an easy process, as many factors influence the success of this strategy. In the context of this research, we utilize a first-order model to show what are the main underlying phenomena that affect transferability and suggest best practices to create adversarial examples that transfer well to the target models.

Software Defect Prediction Using Neural Network Based SMOTE

Software defect prediction is a practical approach to improve the quality and efficiency of time and costs for software testing by focusing on defect modules. The defect prediction software dataset naturally has a class imbalance problem with very few defective modules compared to non-defective modules. Class imbalance can reduce performance from classification. In this study, we applied the Neural Networks Based Synthetic Minority Over-sampling Technique (SMOTE) to overcome class imbalances in the six NASA datasets. Neural Network based on SMOTE is a combination of Neural Network and SMOTE with each hyperparameters that are optimized using random search. The results use a nested 5-cross validation show increases Bal by 25.48% and Recall by 45.99% compared to the original Neural Network. We also compare the performance of Neural Network based SMOTE with SMOTE + Traditional Machine Learning Algorithm. The Neural Network based SMOTE takes first place in the average rank

Utilizing public repositories to improve the decision process for security defect resolution and information reuse in the development environment

Security risks are contained in solutions in software systems that could have been avoided if the design choices were analyzed by using public information security data sources. Public security sources have been shown to contain more relevant and recent information on current technologies than any textbook or research article, and these sources are often used by developers for solving software related problems. However, solutions copied from public discussion forums such as StackOverflow may contain security implications when copied directly into the developers environment. Several different methods to identify security bugs are being implemented, and recent efforts are looking into identifying security bugs from communication artifacts during software development lifecycle as well as using public security information sources to support secure design and development. The primary goal of this thesis is to investigate how to utilize public information sources to reduce security defects in software artifacts through improving the decision process for defect resolution and information reuse in the development environment. We build a data collection tool for collecting data from public information security sources and public discussion forums, construct machine learning models for classifying discussion forum posts and bug reports as security or not-security related, as well as word embedding models for finding matches between public security sources and public discussion forum posts or bug reports. The results of this thesis demonstrate that using public information security sources can provide additional validation layers for defect classification models, as well as provide additional security context for public discussion forum posts. The contributions of this thesis are to provide understanding of how public information security sources can better provide context for bug reports and discussion forums. Additionally, we provide data collection APIs for collecting datasets from these sources, and classification and word embedding models for recommending related security sources for bug reports and public discussion forum posts.Masteroppgave i Programutvikling samarbeid med HVLPROG399MAMN-PRO

Fast scan, an improved approach using machine learning for vulnerability identification

Dissertação de mestrado integrado em Engenharia InformáticaThis document presents a Master Thesis in the Integrated Master’s in Informatics Engi neering focused on the automatic identification of vulnerabilities, that was accomplished at Universidade do Minho in Braga, Portugal. This thesis work aims at developing a machine learning based tool for automatic iden tification of vulnerabilities on programs (source, high level code), that uses an abstract syntax11tree representation. It is based on FastScan, using code2seq approach. Fastscan is a recently developed system aimed capable of detecting vulnerabilities in source code using machine learning techniques. Nevertheless, FastScan is not able of identifying the vulnerability type. In the presented work the main goal is to go further and develop a method to identify specific types of vulnerabilities. As will be shown, the goal will be achieved by changing the method of receiving and processing in a different way the input data and developing an architecture that brings together multiple models to predict different specific vulnerabilities. The best f1 metric obtained is 93% resulting in a precision of 90% and accuracy of 85%, according to the performed tests and regarding a trained model to predict vulnerabilities of the injection type. These results were obtained with the contribution given by the optimization of the model’s hyperparameters and also the use of the Search Cluster from University of Minho that greatly diminished the necessary time to perform training and testing. It is important to refer that overfitting was detected in the late stages of the tests, so this results do not represent the true value in real context. Also an interface is presented, it allows to better interact with the models and analyse the scan results.Este documento apresenta uma dissertação do Mestrado Integrado em Engenharia Infor mática, que tem como foco a automação da deteção de vulnerabilidades e foi concluída na Universidade do Minho em Braga, Portugal. O trabalho apresentado nesta tese pretende desenvolver uma ferramenta que utiliza machine learning e que seja capaz de identificar vulnerabilidades em código. Utilizando para isso a representação do mesmo numa abstract syntax tree. Tem como base FastScan que utiliza a abordagem do code2seq. Fastscan é um projeto recentemente desenvolvido que é capaz de detetar vulnerabilidades em código utilizando técnicas de machine learning, sendo que tem algumas lacunas como o facto de não ser capaz de identificar vulnerabilidades específicas. No trabalho apresentado o objetivo é ir mais além e desenvolver um método capaz de identificar qual o tipo específico de vulnerabilidade presente. Como será apresentado ao longo do documento, este objetivo será alcançado pela alteração do método de receção e processamento dos dados recebidos, assim como o desenvolvimento de uma arquitetura que junte os vários modelos de maneira a cooperarem e a ferramenta ser capaz de detetar e prever a presença de vulnerabilidades específicas. A melhor métrica de f1 obtida foi de 93%, com precisão de 90% e accuracy de 85%, de acordo com os testes efetuados sobre um modelo treinado para prever a presença de vulnerabilidades do tipo de injection. Os resultados foram obtidos devido à otimização dos hiper-parâmetros dos modelos e o cluster Search da Universidade do Minho diminuiu consideravelmente o tempo necessário para efetuar o traino e testes dos modelos. É importante referir que foi detetado overfitting na fase final do desenvolvimento deste trabalho, sendo que os resultados apresentados não representam o valor real dos modelos em contexto real. Para além disso é apresentada uma interface que permite interagir e analisar os resultados de um scan feito pelos modelos