On the Detection of Image-Scaling Attacks in Machine Learning

Image scaling is an integral part of machine learning and computer vision systems. Unfortunately, this preprocessing step is vulnerable to so-called image-scaling attacks where an attacker makes unnoticeable changes to an image so that it becomes a new image after scaling. This opens up new ways for attackers to control the prediction or to improve poisoning and backdoor attacks. While effective techniques exist to prevent scaling attacks, their detection has not been rigorously studied yet. Consequently, it is currently not possible to reliably spot these attacks in practice. This paper presents the first in-depth systematization and analysis of detection methods for image-scaling attacks. We identify two general detection paradigms and derive novel methods from them that are simple in design yet significantly outperform previous work. We demonstrate the efficacy of these methods in a comprehensive evaluation with all major learning platforms and scaling algorithms. First, we show that image-scaling attacks modifying the entire scaled image can be reliably detected even under an adaptive adversary. Second, we find that our methods provide strong detection performance even if only minor parts of the image are manipulated. As a result, we can introduce a novel protection layer against image-scaling attacks.Comment: Accepted at ACSAC'2

Lessons Learned on Machine Learning for Computer Security

We identify 10 generic pitfalls that can affect the experimental outcome of AI driven solutions in computer security. We find that they are prevalent in the literature and provide recommendations for overcoming them in the future

Dos and Don'ts of Machine Learning in Computer Security

With the growing processing power of computing systems and the increasing availability of massive datasets, machine learning algorithms have led to major breakthroughs in many different areas. This development has influenced computer security, spawning a series of work on learning-based security systems, such as for malware detection, vulnerability discovery, and binary code analysis. Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance and render learning-based systems potentially unsuitable for security tasks and practical deployment. In this paper, we look at this problem with critical eyes. First, we identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We conduct a study of 30 papers from top-tier security conferences within the past 10 years, confirming that these pitfalls are widespread in the current security literature. In an empirical analysis, we further demonstrate how individual pitfalls can lead to unrealistic performance and interpretations, obstructing the understanding of the security problem at hand. As a remedy, we propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible. Furthermore, we identify open problems when applying machine learning in security and provide directions for further research.Comment: to appear at USENIX Security Symposium 202

Über die Sicherheit maschinellen Lernens jenseits des Merkmalsraums

Machine learning is increasingly used in security-critical applications, such as malware detection, face recognition, and autonomous driving. However, learning methods are vulnerable to different types of attacks that thwart their secure application. So far, most research has focused on attacks in the feature space of machine learning, that is, the vector space underlying the learning process. Although this has led to a thorough understanding of the possible attack surface, considering the feature space alone ignores the environment machine learning is applied in. Inputs are usually given as real-world objects from a problem space, such as malicious code or PDF files. Hence, an adversary has to consider both the problem and the feature space. This is not trivial, as both spaces have no one-to-one relation in most application areas and feature-space attacks are thus not directly applicable in practice. As a result, a more thorough examination is required to understand the real-world impact of attacks against machine learning. In this thesis, we explore the relation between the problem and the feature space regarding the attack surface of learning-based systems. First, we analyze attacks in the problem space that create real objects and that mislead learning methods in the feature space. A framework is developed to examine the challenges, constraints, and search strategies. To gain practical insights, we examine a problem-space attack against source code attribution. The created adversarial examples mislead the attribution in the majority of cases. Second, we analyze the mapping from problem to feature space. Using the example of image scaling, we study attacks that exploit the mapping and that are agnostic to the learning model or training data. After identifying the root cause of these attacks, defenses for prevention are examined against adversaries of different strengths. Furthermore, the feature space also has an inherent connection to the media space of digital watermarking. This space is a vector space in which watermarks are embedded and detected. As adversaries target this process, attacks and defenses have been extensively studied here as well. Linking both spaces allows us to transfer attacks, defenses, and knowledge between machine learning and watermarking. Taken together, this thesis provides a novel view on the security of machine learning beyond the feature space by including the problem space and the media space into the security analysis.Maschinelles Lernen wird zunehmend in sicherheitskritischen Anwendungen eingesetzt, zum Beispiel im Bereich der Schadsoftware-Erkennung, der Gesichtserkennung und des autonomen Fahrens. Allerdings können Angreifer Lernmethoden selbst gezielt umgehen oder täuschen. Hierbei hat sich bislang ein Großteil der Forschung auf Angriffe im Merkmalsraum von Lernmethoden beschränkt. In diesem Vektorraum findet der Lernprozess statt, sodass der Fokus auf diesen Raum zu einem soliden Verständnis über die Angriffsfläche im maschinellen Lernen geführt hat. Jedoch ist die alleinige Betrachtung des Merkmalsraums nicht ausreichend. In der Regel bestehen die Eingaben im maschinellen Lernen aus realen Objekten aus einem Problemraum, wie beispielsweise schädlichen Programmcode- oder PDF-Dateien. Ein Angreifer muss daher sowohl diesen Problemraum als auch den Merkmalsraum berücksichtigen. Dies ist nicht trivial, da beide Räume häufig keine 1:1-Beziehung aufweisen und somit Angriffe aus dem Merkmalsraum in der Praxis nicht direkt anwendbar sind. Um ein besseres Verständnis über die realen Auswirkungen möglicher Angriffe im maschinellen Lernen zu erlangen, sind daher weiterführende Untersuchungen nötig. Diese Dissertation untersucht dazu die Beziehung zwischen Problemraum und Merkmalsraum hinsichtlich der Angriffsfläche lernbasierter Systeme. Es werden zuerst Angriffe im Problemraum betrachtet, welche reale Objekte erzeugen und gleichzeitig Lernmethoden im Merkmalsraum täuschen. Die mit dem Angriff verbundenen Herausforderungen, Nebenbedingungen und Suchstrategien werden hierbei systematisch festgehalten. Die gewonnenen Erkenntnisse werden praktisch am Beispiel eines Angriffs gegen Identifikationsmethoden, welche Entwickler basierend auf Programmcode erkennen, eingesetzt. Als zweiter Kernpunkt der Analyse wird konkret die Abbildung aus dem Problemraum in den Merkmalsraum betrachtet. Am Beispiel von Bildskalierungen wird ein Angriff untersucht, welcher die Vorverarbeitung in dieser Abbildung gezielt ausnutzt. Der Angriff hängt somit weder vom Lernmodell noch von den Trainingsdaten ab. Eine Analyse der Angriffsursachen führt zur Entwicklung mehrerer Verteidigungsstrategien, die einen Angriff präventiv verhindern. Abschließend stellt diese Dissertation eine Abbildung zwischen dem Merkmalsraum aus dem maschinellen Lernen und dem Medienraum digitaler Wasserzeichenverfahren her. Dies erlaubt den Transfer von Angriffen, Verteidigungen und Erkenntnissen aus beiden Forschungsdisziplinen

Misleading Deep-Fake Detection with GAN Fingerprints

Generative adversarial networks (GANs) have made remarkable progress in synthesizing realistic-looking images that effectively outsmart even humans. Although several detection methods can recognize these deep fakes by checking for image artifacts from the generation process, multiple counterattacks have demonstrated their limitations. These attacks, however, still require certain conditions to hold, such as interacting with the detection method or adjusting the GAN directly. In this paper, we introduce a novel class of simple counterattacks that overcomes these limitations. In particular, we show that an adversary can remove indicative artifacts, the GAN fingerprint, directly from the frequency spectrum of a generated image. We explore different realizations of this removal, ranging from filtering high frequencies to more nuanced frequency-peak cleansing. We evaluate the performance of our attack with different detection methods, GAN architectures, and datasets. Our results show that an adversary can often remove GAN fingerprints and thus evade the detection of generated images.Comment: In IEEE Deep Learning and Security Workshop (DLS) 202

A Synthetic Transcriptional Activator of Genes Associated with the Retina in Human Dermal Fibroblasts

Small molecules capable of modulating epigenetic signatures can activate the transcription of tissue-restricted genes in a totally unrelated cell type and have potential use in epigenetic therapy. To provide an example for an initial approach, we report here on one synthetic small-molecule compound-termed "SAHA-PIP X"-from our library of conjugates. This compound triggered histone acetylation accompanied by the transcription of retinal-tissue-related genes in human dermal fibroblasts (HDFs)