Benchmarking Perturbation-based Saliency Maps for Explaining Deep Reinforcement Learning Agents

Recent years saw a plethora of work on explaining complex intelligent agents. One example is the development of several algorithms that generate saliency maps which show how much each pixel attributed to the agents' decision. However, most evaluations of such saliency maps focus on image classification tasks. As far as we know, there is no work which thoroughly compares different saliency maps for Deep Reinforcement Learning agents. This paper compares four perturbation-based approaches to create saliency maps for Deep Reinforcement Learning agents trained on four different Atari 2600 games. All four approaches work by perturbing parts of the input and measuring how much this affects the agent's output. The approaches are compared using three computational metrics: dependence on the learned parameters of the agent (sanity checks), faithfulness to the agent's reasoning (input degradation), and run-time.Comment: Presented on the Explainable Agency in Artificial Intelligence Workshop during the 35th AAAI Conference on Artificial Intelligenc

Effizientes Netzwerkmonitoring für Angriffserkennung

Techniques for network-based intrusion detection have been evolving for years, and the focus of most research is on detection algorithms, although networks are distributed and dynamically managed nowadays. A data processing framework is required that allows to embed multiple detection techniques and to provide data with the needed aggregation levels. Within that framework, this work concentrates on methods that improve the interoperability of intrusion detection techniques and focuses on data preprocessing stages that perform data evaluation and intelligent data filtering. After presenting a survey of the chain of processes needed for network-based intrusion detection, I discuss the evaluation of TCP connection states based on aggregated flow data. I develop classifiers that interpret flow data in regard of failed and successful connections. These classifiers are especially relevant for anomaly-based intrusion detection techniques like port scan or malware detection, and enable many of these techniques to operate on flow-level data instead of packet-level data. The second part focuses on the filtering of payload data for IDSs that use signatures for detection. I perform a detailed analysis of the IDS Snort that locates specific patterns within connections. This analysis led to the first approach, FPA (Front Payload Aggregation), which captures data that is transferred at the beginning of connections. Unfortunately, interleaved communication patterns cannot be captured well using this aggregation technique. Therefore I propose DPA (Dialog-based Payload Aggregation) in the next part, which divides bidirectional communication into dialog segments. For each direction change in the communication, a certain amount of transferred data is kept, and the rest is dropped. This way, bulk data is dropped using a very lightweight method that only relies on network and transport header information. The filter achieved very good results in combination with the IDS Snort, as 89% of the original events could be retained, whereas only 4% of the original amount of data was analyzed by the IDS. To exploit the multi-core architecture of today's CPUs, IDSs are executed in parallel and a load balancer distributes data to the systems. As payload-based analysis is not able to cope with current network speeds even with parallelization, I develop an approach to perform intelligent selection of the captured network data and to distribute selected data to multiple IDSs. The selection algorithm is based on a priority system that keeps track of each host's monitored time and the system controls data losses by monitoring the load of every IDS. My evaluation revealed that the system showed up to 40% better detection results compared to an overloaded system that dropped the same amount of packets in an uncontrolled way due to overload.Eine steigende Anzahl an Einbrüchen und Schadanwendungen im Internet zeigt, dass nicht alle verbundenen Rechensysteme ausreichend durch Sicherheitsmaßnahmen geschützt werden. Deswegen ist es nötig, in Netzwerken verdächtige Datenströme zu erkennen welche Teil eines Angriffs sind oder zu Schadanwendungen gehören. Diese Dissertation schlägt ein System vor, das für diese Aufgabe ein Rahmenwerk zur Datenverarbeitung bereitstellt. Dieses Rahmenwerk ist fähig, Information auf verschiedenen Aggregationsstufen für diverse Erkennungstechniken zur Verfügung zu stellen. Dabei konzentriere ich mich auf die Verbesserung der Zusammenarbeit unterschiedlicher Techniken der Angriffserkennung und behandle vor allem die Vorverarbeitung und intelligente Filterung von Daten. Im ersten Teil der Arbeit gebe ich einen Überblick über die gesamte Prozesskette der netzwerkbasierten Angriffserkennung. Anschließend wird die Auswertung von Verbindungszuständen auf Basis flowbasierter Daten behandelt. Ich entwerfe Klassifikatoren, welche aggregierte Flowdaten hinsichtlich fehlgeschlagener und erfolgreicher Verbindungen auswerten. Diese Klassifikatoren sind besonders relevant für anomaliebasierte Methoden zur Angriffserkennung und erlauben es, dass viele dieser Methoden nicht nur mit Hilfe detaillierter Paketdaten angewendet werden können, sondern auch mit aggregierten Flowdaten. Weiterhin wird die Vorfilterung von Paketinhalten für Angriffserkennungssysteme in dieser Arbeit behandelt. Ich führe eine detaillierte Analyse des Angriffserkennungssystems Snort durch, welches Mustererkennung auf Paketinhalten durchführt. Diese Analyse führt zum ersten Ansatz, der FPA (Front Payload Aggregation). Diese Technik extrahiert Daten, die sich am Anfang von Verbindungen befinden. Leider werden verschachtelte Kommunikationsmuster nur unzureichend durch diese Filterungsmethode erfasst. Deswegen erweitere ich FPA zur DPA (Dialog-based Payload Aggregation), welche bidirektionale Kommunikation in Dialogsegmente unterteilt. Bei jedem Richtungswechsel in der Kommunikation wird dabei eine definierte Menge an Payloaddaten aufgezeichnet, und der Rest wird verworfen. DPA erreichte sehr gute Ergebnisse in Kombination mit dem Angriffserkennungssystem Snort, welches 89% aller vorhandenen Ereignisse erkannte, obwohl 96% der originalen Datenmenge von DPA ausgefiltert wurden. Um Mehrkernarchitekturen von neuen Prozessoren auszunutzen, werden mittlerweile netzwerkbasierte Angriffserkennungssysteme parallel ausgeführt, und ein Lastverteiler leitet Daten zu diesen Systemen. Da auf Paketinhalten basierende Erkennungssysteme für aktuelle Netzwerkgeschwindigkeiten zu langsam arbeiten, entwerfe ich eine Methode für die intelligente Auswahl von empfangenen Netzwerkpaketen. Nur ein Teil der empfangenen Pakete wird dabei zu Angriffserkennungssystemen weitergeleitet, und das System kontrolliert Datenverluste, indem Überlast in den Erkennungssystemen vermieden wird. Für eine ausgeglichene Auswahl ist ein Prioritätsmodell im System integriert, das die Beobachtungszeit einzelner Rechnern im lokalen Netzwerk verfolgt. Die Auswertung zeigte, dass das System bis zu 40% bessere Erkennungsraten hatte als ein überlastetes System, das die gleiche Datenmenge wegen Überlast unkontrolliert verlor

Benchmarking perturbation-based saliency maps for explaining Atari agents

One of the most prominent methods for explaining the behavior of Deep Reinforcement Learning (DRL) agents is the generation of saliency maps that show how much each pixel attributed to the agents' decision. However, there is no work that computationally evaluates and compares the fidelity of different perturbation-based saliency map approaches specifically for DRL agents. It is particularly challenging to computationally evaluate saliency maps for DRL agents since their decisions are part of an overarching policy, which includes long-term decision making. For instance, the output neurons of value-based DRL algorithms encode both the value of the current state as well as the expected future reward after doing each action in this state. This ambiguity should be considered when evaluating saliency maps for such agents. In this paper, we compare five popular perturbation-based approaches to create saliency maps for DRL agents trained on four different Atari 2,600 games. The approaches are compared using two computational metrics: dependence on the learned parameters of the underlying deep Q-network of the agents (sanity checks) and fidelity to the agents' reasoning (input degradation). During the sanity checks, we found that a popular noise-based saliency map approach for DRL agents shows little dependence on the parameters of the output layer. We demonstrate that this can be fixed by tweaking the algorithm such that it focuses on specific actions instead of the general entropy within the output values. For fidelity, we identify two main factors that influence which saliency map approach should be chosen in which situation. Particular to value-based DRL agents, we show that analyzing the agents' choice of action requires different saliency map approaches than analyzing the agents' state value estimation

Dialog-based Payload Aggregation for Intrusion Detection

Network-based Intrusion Detection Systems (IDSs) such as Snort or Bro that have to analyze the packet payload for all the received data show severe performance problems if used in high-speed networks. Recent research results improve pattern matchers based on efficient algorithms or using specialized hardware. We approach the problem in a completely different way by considerably reducing the amount of data to be analyzed with only marginal impact on the detection quality. Dialog-based Payload Aggregation (DPA) uses TCP sequence numbers to decide which parts of the payload need to be analyzed by the IDS. Whenever a connection starts, or if the direction of the data transmission between peers changes, we forward the next N bytes of traffic to an attached IDS. All data transferred after the window is discarded. Our analysis using live network traffic and multiple Snort rulesets shows that most of the pattern matches occur at the beginning of connections or directly after direction changes in the data streams. According to our experimental results, our method reduces the data rate to be processed to around 1 % in a typical network while retaining more than 98 % of all detected events. Assuming a linear relationship between the data rate and processing time of an IDS, this results in a speedup of two magnitudes in the best case

Flow-based TCP Connection Analysis

We discuss the need for accurate analysis of TCP connections based on aggregated flow information. Due to increasing bandwidths in the Internet, flow metering is thought to be the a promising solution for network monitoring, because packet-oriented state-based analysis reaches its limits and fast hardware support for flow metering is already integrated in modern routers. Motivated by earlier work on flow-based connection analysis, we investigate the quality of several stateless classifiers that can be used to determine the TCP connection state as either successful or failed. This information is strongly needed especially in the domain of attack detection and is usually produced by fine-grained analysis in the packet level. Furthermore, we determine appropriate configuration parameters for optimal flow metering by introducing a new statistical property, the maximum packet gap. We evaluated both, the classifiers and the packet gap analysis using a number of representative packet traces. Our best classifiers are able to correctly identify 95 % of all connections with a fraction of the processing costs required for packet-based stateful connection tracking

Flow-based Front Payload Aggregation

We present and discuss a new monitoring technique that we call Front Payload Aggregation (FPA). Instead of being limited to either analyzing single packets for signature-based attack detection or exploiting statistical flow information for anomaly detection, FPA combines the advantages of both approaches. Exploiting the fact that most attack signatures can be found in the very first packets of a connection, we collect payload information from these few packets (we take the first n payload Bytes) and associate it to the corresponding flow data. Thus, intrusion detection can still be performed with a high degree of confidence and the monitoring system becomes efficient w.r.t. processing performance and attack resilience