Security Evaluation of Support Vector Machines in Adversarial Environments

Support Vector Machines (SVMs) are among the most popular classification techniques adopted in security applications like malware detection, intrusion detection, and spam filtering. However, if SVMs are to be incorporated in real-world security systems, they must be able to cope with attack patterns that can either mislead the learning algorithm (poisoning), evade detection (evasion), or gain information about their internal parameters (privacy breaches). The main contributions of this chapter are twofold. First, we introduce a formal general framework for the empirical evaluation of the security of machine-learning systems. Second, according to our framework, we demonstrate the feasibility of evasion, poisoning and privacy attacks against SVMs in real-world security problems. For each attack technique, we evaluate its impact and discuss whether (and how) it can be countered through an adversary-aware design of SVMs. Our experiments are easily reproducible thanks to open-source code that we have made available, together with all the employed datasets, on a public repository.Comment: 47 pages, 9 figures; chapter accepted into book 'Support Vector Machine Applications

Joint Detection-Estimation Games for Sensitivity Analysis Attacks

ABSTRACT Sensitivity analysis attacks aim at estimating a watermark from multiple observations of the detector's output. Subsequently, the attacker removes the estimated watermark from the watermarked signal. In order to measure the vulnerability of a detector against such attacks, we evaluate the fundamental performance limits for the attacker's estimation problem. The inverse of the Fisher information matrix provides a bound on the covariance matrix of the estimation error. A general strategy for the attacker is to select the distribution of auxiliary test signals that minimizes the trace of the inverse Fisher information matrix. The watermark detector must trade off two conflicting requirements: (1) reliability, and (2) security against sensitivity attacks. We explore this tradeoff and design the detection function that maximizes the trace of the attacker's inverse Fisher information matrix while simultaneously guaranteeing a bound on the error probability. Game theory is the natural framework to study this problem, and considerable insights emerge from this analysis

A Comparative Analysis of Adversarial Capabilities, Attacks, and Defenses Across the Machine Learning Pipeline in White-Box and Black-Box Settings

The increasing adoption of machine learning models across various domains has brought to light the critical issue of their vulnerability to adversarial attacks, raising concerns about their security and reliability. This research discusses the adversarial capabilities that can be exploited at different stages of the machine learning pipeline: training, testing, and deployment. We investigate the distinct challenges and opportunities adversaries face in both transparent (white-box) and opaque (black-box) settings. Adversaries can tamper with the training data during the initial phase of the machine learning pipeline, compromising the model\u27s learning process. In a transparent setup, adversaries possess the ability to directly alter, introduce, or eliminate samples, injecting malicious patterns. Conversely, in an opaque setup, adversaries can indirectly sway the training process by manipulating data collection or preprocessing stages. Safeguarding against training-stage attacks necessitates data cleansing, anomaly identification, and resilient training techniques like adversarial training. Moving on to the testing phase, adversaries concentrate on designing deceptive examples that mislead the trained model. Adversaries with comprehensive knowledge of the model, operating in a white-box scenario, can meticulously craft highly targeted adversarial instances. On the other hand, black-box adversaries, lacking direct access to the model, employ techniques such as transferability to generate adversarial examples. Effective countermeasures against testing-stage attacks encompass adversarial training, ensemble approaches, and randomized defense mechanisms. As the model is deployed and used in real-world contexts, adversaries exploit vulnerabilities to undermine its performance. In a white-box setting, adversaries can meticulously examine the model\u27s behavior and engineer targeted attacks. Conversely, black-box adversaries probe the model and exploit weaknesses by carefully constructing malicious inputs. To protect against deployment-stage threats, defenses such as real-time monitoring, anomaly detection, secure deployment practices, and regular security assessments are also discussed

Roadmap on optical security

Postprint (author's final draft

Dynamic adversarial mining - effectively applying machine learning in adversarial non-stationary environments.

While understanding of machine learning and data mining is still in its budding stages, the engineering applications of the same has found immense acceptance and success. Cybersecurity applications such as intrusion detection systems, spam filtering, and CAPTCHA authentication, have all begun adopting machine learning as a viable technique to deal with large scale adversarial activity. However, the naive usage of machine learning in an adversarial setting is prone to reverse engineering and evasion attacks, as most of these techniques were designed primarily for a static setting. The security domain is a dynamic landscape, with an ongoing never ending arms race between the system designer and the attackers. Any solution designed for such a domain needs to take into account an active adversary and needs to evolve over time, in the face of emerging threats. We term this as the ‘Dynamic Adversarial Mining’ problem, and the presented work provides the foundation for this new interdisciplinary area of research, at the crossroads of Machine Learning, Cybersecurity, and Streaming Data Mining. We start with a white hat analysis of the vulnerabilities of classification systems to exploratory attack. The proposed ‘Seed-Explore-Exploit’ framework provides characterization and modeling of attacks, ranging from simple random evasion attacks to sophisticated reverse engineering. It is observed that, even systems having prediction accuracy close to 100%, can be easily evaded with more than 90% precision. This evasion can be performed without any information about the underlying classifier, training dataset, or the domain of application. Attacks on machine learning systems cause the data to exhibit non stationarity (i.e., the training and the testing data have different distributions). It is necessary to detect these changes in distribution, called concept drift, as they could cause the prediction performance of the model to degrade over time. However, the detection cannot overly rely on labeled data to compute performance explicitly and monitor a drop, as labeling is expensive and time consuming, and at times may not be a possibility altogether. As such, we propose the ‘Margin Density Drift Detection (MD3)’ algorithm, which can reliably detect concept drift from unlabeled data only. MD3 provides high detection accuracy with a low false alarm rate, making it suitable for cybersecurity applications; where excessive false alarms are expensive and can lead to loss of trust in the warning system. Additionally, MD3 is designed as a classifier independent and streaming algorithm for usage in a variety of continuous never-ending learning systems. We then propose a ‘Dynamic Adversarial Mining’ based learning framework, for learning in non-stationary and adversarial environments, which provides ‘security by design’. The proposed ‘Predict-Detect’ classifier framework, aims to provide: robustness against attacks, ease of attack detection using unlabeled data, and swift recovery from attacks. Ideas of feature hiding and obfuscation of feature importance are proposed as strategies to enhance the learning framework\u27s security. Metrics for evaluating the dynamic security of a system and recover-ability after an attack are introduced to provide a practical way of measuring efficacy of dynamic security strategies. The framework is developed as a streaming data methodology, capable of continually functioning with limited supervision and effectively responding to adversarial dynamics. The developed ideas, methodology, algorithms, and experimental analysis, aim to provide a foundation for future work in the area of ‘Dynamic Adversarial Mining’, wherein a holistic approach to machine learning based security is motivated