149 research outputs found
Evaluating Explanation Methods for Deep Learning in Security
Deep learning is increasingly used as a building block of security systems.
Unfortunately, neural networks are hard to interpret and typically opaque to
the practitioner. The machine learning community has started to address this
problem by developing methods for explaining the predictions of neural
networks. While several of these approaches have been successfully applied in
the area of computer vision, their application in security has received little
attention so far. It is an open question which explanation methods are
appropriate for computer security and what requirements they need to satisfy.
In this paper, we introduce criteria for comparing and evaluating explanation
methods in the context of computer security. These cover general properties,
such as the accuracy of explanations, as well as security-focused aspects, such
as the completeness, efficiency, and robustness. Based on our criteria, we
investigate six popular explanation methods and assess their utility in
security systems for malware detection and vulnerability discovery. We observe
significant differences between the methods and build on these to derive
general recommendations for selecting and applying explanation methods in
computer security.Comment: IEEE European Symposium on Security and Privacy, 202
Machine Unlearning of Features and Labels
Removing information from a machine learning model is a non-trivial task that
requires to partially revert the training process. This task is unavoidable
when sensitive data, such as credit card numbers or passwords, accidentally
enter the model and need to be removed afterwards. Recently, different concepts
for machine unlearning have been proposed to address this problem. While these
approaches are effective in removing individual data points, they do not scale
to scenarios where larger groups of features and labels need to be reverted. In
this paper, we propose the first method for unlearning features and labels. Our
approach builds on the concept of influence functions and realizes unlearning
through closed-form updates of model parameters. It enables to adapt the
influence of training data on a learning model retrospectively, thereby
correcting data leaks and privacy issues. For learning models with strongly
convex loss functions, our method provides certified unlearning with
theoretical guarantees. For models with non-convex losses, we empirically show
that unlearning features and labels is effective and significantly faster than
other strategies.Comment: Network and Distributed System Security Symposium (NDSS) 202
Untersuchung und Klassifikation der Fahreraufmerksamkeit bei längerer partiell automatisierter Fahrt
Um die Unfallzahlen weiter zu senken, schreibt die Europäische Union ab 2030 eine höhere Fahrerüberwachung für neue Fahrzeuge vor. Bislang liegt der Fokus in einem manuell gefahrenen Fahrzeug auf einer Müdigkeitserkennung als Komfortsystem. Jedoch ändern sich die Anforderungen an den Fahrer und dessen Aufgaben bei steigender Automatisierung der Fahraufgabe, sodass auch die Fahrerbeobachtung/-überwachung weiter entwickelt werden sollte. Die vorliegende Arbeit beschäftigt sich mit der Auslegung von Aufmerksamkeitsaufgaben in Simulator-Fahrversuchen und einem ersten Versuch die aufgenommenen Daten von wenigen Probanden automatisch auszuwerten, um die Machbarkeit zu zeigen. Durch die partielle Automatisierung (nach SAE Level 2) der Fahraufgabe bleiben dem Fahrer bei der hier vorgestellten längeren Aufmerksamkeitsaufgabe nur noch schwache und seltene Reize, sodass von einer Vigilanzaufgabe gesprochen werden kann. Zur Messung des Fahrerzustands wird in dieser Machbarkeitsstudie eine Kombination aus verschieden Körperfunktionen durch unterschiedliche Sensoren erfasst, mit einem künstlichen neuronalen Netzwerk fusioniert und verarbeitet. Es werden verschiedene Kombinationen von Sensoren getestet, um herauszufinden, welche Signale für eine Klassifikation des Fahrerzustands ausreichend oder notwendig sind. Dabei wird darauf geachtet, ob diese Signale in Zukunft durch neue Sensortechnologien auch kontaktlos erfasst werden könnten, um eine Entwicklung dieser zu motivieren. Am Ende wird eine Klassifizierung vorgenommen, die zwischen den hier so genannten Zuständen Vigilant (aufmerksam) und Hypovigilant (nicht mehr aufmerksam) unterscheiden kann, sodass eine anschließende Regelung ein solches Signal verwenden kann, um den Fahrer in einen besseren Zustand zu bringen. Zudem kann der Fahrer gewarnt werden, wenn die Aufmerksamkeit nicht zu den Aufgaben passt, die er in der Automatisierungsstufe zu erledigen hat beziehungsweise für die er verantwortlich ist. Dazu könnten Aufgaben zwischen dem Fahrzeug und dem Fahrer dynamisch verteilt werden, also beispielsweise die Führung in Längs- oder Querrichtung wieder als Fahraufgabe dem Fahrer übergeben werden, um ihn aufmerksam zu halten. Durch eine kontaktlose Erfassung der Aufmerksamkeit gibt es zudem einen Komfortgewinn, da zum Beispiel die Hände nicht mehr das Lenkrad festhalten müssen, um dem Fahrzeug seine Aufmerksamkeit zu bestätigen, wie es heute in einigen Fahrzeugen üblich ist
Evil from Within: Machine Learning Backdoors through Hardware Trojans
Backdoors pose a serious threat to machine learning, as they can compromise
the integrity of security-critical systems, such as self-driving cars. While
different defenses have been proposed to address this threat, they all rely on
the assumption that the hardware on which the learning models are executed
during inference is trusted. In this paper, we challenge this assumption and
introduce a backdoor attack that completely resides within a common hardware
accelerator for machine learning. Outside of the accelerator, neither the
learning model nor the software is manipulated, so that current defenses fail.
To make this attack practical, we overcome two challenges: First, as memory on
a hardware accelerator is severely limited, we introduce the concept of a
minimal backdoor that deviates as little as possible from the original model
and is activated by replacing a few model parameters only. Second, we develop a
configurable hardware trojan that can be provisioned with the backdoor and
performs a replacement only when the specific target model is processed. We
demonstrate the practical feasibility of our attack by implanting our hardware
trojan into the Xilinx Vitis AI DPU, a commercial machine-learning accelerator.
We configure the trojan with a minimal backdoor for a traffic-sign recognition
system. The backdoor replaces only 30 (0.069%) model parameters, yet it
reliably manipulates the recognition once the input contains a backdoor
trigger. Our attack expands the hardware circuit of the accelerator by 0.24%
and induces no run-time overhead, rendering a detection hardly possible. Given
the complex and highly distributed manufacturing process of current hardware,
our work points to a new threat in machine learning that is inaccessible to
current security mechanisms and calls for hardware to be manufactured only in
fully trusted environments
Probing interneuronal cell communication via optogenetic stimulation
This study uses an all-optical approach to probe interneuronal communication between spiral ganglion neurons (SGNs) and neurons of other functional units, in this case cortex neurons (CNs) and hippocampus neurons (HNs), for the first time. We combined a channelrhodopsin variant (CheRiff) with a red genetically encoded calcium indicator (jRCaMP1a), enabling simultaneous optical stimulation and recording from spatially separated small neuronal populations. Stimulation of SGNs was possible with both optogenetic manipulated HNs and CNs, respectively. Furthermore, a dependency on the pulse duration of the stimulating light in regard to the evoked calcium response in the SGNs was also observed. Our results pave the way to enable innovative technologies based on “biohybrid” systems utilizing the functional interaction between different biological (eg, neural) systems. This can enable improved treatment of neurological and sensorineural disorders such as hearing loss
Real-Time Radar-Based Gesture Detection and Recognition Built in an Edge-Computing Platform
In this paper, a real-time signal processing frame-work based on a 60 GHz
frequency-modulated continuous wave (FMCW) radar system to recognize gestures
is proposed. In order to improve the robustness of the radar-based gesture
recognition system, the proposed framework extracts a comprehensive hand
profile, including range, Doppler, azimuth and elevation, over multiple
measurement-cycles and encodes them into a feature cube. Rather than feeding
the range-Doppler spectrum sequence into a deep convolutional neural network
(CNN) connected with recurrent neural networks, the proposed framework takes
the aforementioned feature cube as input of a shallow CNN for gesture
recognition to reduce the computational complexity. In addition, we develop a
hand activity detection (HAD) algorithm to automatize the detection of gestures
in real-time case. The proposed HAD can capture the time-stamp at which a
gesture finishes and feeds the hand profile of all the relevant
measurement-cycles before this time-stamp into the CNN with low latency. Since
the proposed framework is able to detect and classify gestures at limited
computational cost, it could be deployed in an edge-computing platform for
real-time applications, whose performance is notedly inferior to a
state-of-the-art personal computer. The experimental results show that the
proposed framework has the capability of classifying 12 gestures in real-time
with a high F1-score.Comment: Accepted for publication in IEEE Sensors Journal. A video is
available on https://youtu.be/IR5NnZvZBL
Dos and Don'ts of Machine Learning in Computer Security
With the growing processing power of computing systems and the increasing
availability of massive datasets, machine learning algorithms have led to major
breakthroughs in many different areas. This development has influenced computer
security, spawning a series of work on learning-based security systems, such as
for malware detection, vulnerability discovery, and binary code analysis.
Despite great potential, machine learning in security is prone to subtle
pitfalls that undermine its performance and render learning-based systems
potentially unsuitable for security tasks and practical deployment. In this
paper, we look at this problem with critical eyes. First, we identify common
pitfalls in the design, implementation, and evaluation of learning-based
security systems. We conduct a study of 30 papers from top-tier security
conferences within the past 10 years, confirming that these pitfalls are
widespread in the current security literature. In an empirical analysis, we
further demonstrate how individual pitfalls can lead to unrealistic performance
and interpretations, obstructing the understanding of the security problem at
hand. As a remedy, we propose actionable recommendations to support researchers
in avoiding or mitigating the pitfalls where possible. Furthermore, we identify
open problems when applying machine learning in security and provide directions
for further research.Comment: to appear at USENIX Security Symposium 202
Lessons Learned on Machine Learning for Computer Security
We identify 10 generic pitfalls that can affect the experimental outcome of AI driven solutions in computer security. We find that they are prevalent in the literature and provide recommendations for overcoming them in the future
Scanning laser optical tomography for in toto imaging of the murine cochlea
The mammalian cochlea is a complex macroscopic structure due to its helical shape and the microscopic arrangements of the individual layers of cells. To improve the outcomes of hearing restoration in deaf patients, it is important to understand the anatomic structure and composition of the cochlea ex vivo. Hitherto, only one histological technique based on confocal laser scanning microscopy and optical clearing has been developed for in toto optical imaging of the murine cochlea. However, with a growing size of the specimen, e.g., human cochlea, this technique reaches its limitations. Here, we demonstrate scanning laser optical tomography (SLOT) as a valuable imaging technique to visualize the murine cochlea in toto without any physical slicing. This technique can also be applied in larger specimens up to cm3 such as the human cochlea. Furthermore, immunolabeling allows visualization of inner hair cells (otoferlin) or spiral ganglion cells (neurofilament) within the whole cochlea. After image reconstruction, the 3D dataset was used for digital segmentation of the labeled region. As a result, quantitative analysis of position, length and curvature of the labeled region was possible. This is of high interest in order to understand the interaction of cochlear implants (CI) and cells in more detail. © 2017 Nolte et al.This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.DFG/EXC/1077/1Ministry of Lower SaxonyVolkswagenStiftun
- …