1,552 research outputs found
CSI Neural Network: Using Side-channels to Recover Your Artificial Neural Network Information
Machine learning has become mainstream across industries. Numerous examples
proved the validity of it for security applications. In this work, we
investigate how to reverse engineer a neural network by using only power
side-channel information. To this end, we consider a multilayer perceptron as
the machine learning architecture of choice and assume a non-invasive and
eavesdropping attacker capable of measuring only passive side-channel leakages
like power consumption, electromagnetic radiation, and reaction time.
We conduct all experiments on real data and common neural net architectures
in order to properly assess the applicability and extendability of those
attacks. Practical results are shown on an ARM CORTEX-M3 microcontroller. Our
experiments show that the side-channel attacker is capable of obtaining the
following information: the activation functions used in the architecture, the
number of layers and neurons in the layers, the number of output classes, and
weights in the neural network. Thus, the attacker can effectively reverse
engineer the network using side-channel information.
Next, we show that once the attacker has the knowledge about the neural
network architecture, he/she could also recover the inputs to the network with
only a single-shot measurement. Finally, we discuss several mitigations one
could use to thwart such attacks.Comment: 15 pages, 16 figure
Exploring the Effectiveness of Privacy Preserving Classification in Convolutional Neural Networks
A front-runner in modern technological advancement, machine learning relies heavily on the use of personal data. It follows that, when assessing the scope of confidentiality for machine learning models, understanding the potential role of encryption is critical. Convolutional Neural Networks (CNN) are a subset of artificial feed-forward neural networks tailored specifically for image recognition and classification. As the popularity of CNN increases, so too does the need for privacy preserving classification. Homomorphic Encryption (HE) refers to a cryptographic system that allows for computation on encrypted data to obtain an encrypted result such that, when decrypted, the result is the same value that would have been obtained if the operations were performed on the original unencrypted data. The objective of this research was to explore the application of HE alongside CNN with the creation of privacy-preserving CNN layers that have the ability to operate on encrypted images. This was accomplished through (1) researching the underlying structure of preexisting privacy-preserving CNN classifiers, (2) creating privacy-preserving convolution, pooling, and fully-connected layers by mapping the computations found within each layer to a space of homomorphic computations, (3) developing a polynomial-approximated activation function and creating a privacy-preserving activation layer based on this approximation, (4) testing and profiling the designed application to asses efficiency, performance, accuracy, and overall practicality
SNIFF: Reverse Engineering of Neural Networks with Fault Attacks
Neural networks have been shown to be vulnerable against fault injection
attacks. These attacks change the physical behavior of the device during the
computation, resulting in a change of value that is currently being computed.
They can be realized by various fault injection techniques, ranging from
clock/voltage glitching to application of lasers to rowhammer. In this paper we
explore the possibility to reverse engineer neural networks with the usage of
fault attacks. SNIFF stands for sign bit flip fault, which enables the reverse
engineering by changing the sign of intermediate values. We develop the first
exact extraction method on deep-layer feature extractor networks that provably
allows the recovery of the model parameters. Our experiments with Keras library
show that the precision error for the parameter recovery for the tested
networks is less than with the usage of 64-bit floats, which
improves the current state of the art by 6 orders of magnitude. Additionally,
we discuss the protection techniques against fault injection attacks that can
be applied to enhance the fault resistance
An Experimental Study of Reduced-Voltage Operation in Modern FPGAs for Neural Network Acceleration
We empirically evaluate an undervolting technique, i.e., underscaling the
circuit supply voltage below the nominal level, to improve the power-efficiency
of Convolutional Neural Network (CNN) accelerators mapped to Field Programmable
Gate Arrays (FPGAs). Undervolting below a safe voltage level can lead to timing
faults due to excessive circuit latency increase. We evaluate the
reliability-power trade-off for such accelerators. Specifically, we
experimentally study the reduced-voltage operation of multiple components of
real FPGAs, characterize the corresponding reliability behavior of CNN
accelerators, propose techniques to minimize the drawbacks of reduced-voltage
operation, and combine undervolting with architectural CNN optimization
techniques, i.e., quantization and pruning. We investigate the effect of
environmental temperature on the reliability-power trade-off of such
accelerators. We perform experiments on three identical samples of modern
Xilinx ZCU102 FPGA platforms with five state-of-the-art image classification
CNN benchmarks. This approach allows us to study the effects of our
undervolting technique for both software and hardware variability. We achieve
more than 3X power-efficiency (GOPs/W) gain via undervolting. 2.6X of this gain
is the result of eliminating the voltage guardband region, i.e., the safe
voltage region below the nominal level that is set by FPGA vendor to ensure
correct functionality in worst-case environmental and circuit conditions. 43%
of the power-efficiency gain is due to further undervolting below the
guardband, which comes at the cost of accuracy loss in the CNN accelerator. We
evaluate an effective frequency underscaling technique that prevents this
accuracy loss, and find that it reduces the power-efficiency gain from 43% to
25%.Comment: To appear at the DSN 2020 conferenc
BlackJack: Secure machine learning on IoT devices through hardware-based shuffling
Neural networks are seeing increased use in diverse Internet of Things (IoT)
applications such as healthcare, smart homes and industrial monitoring. Their
widespread use makes neural networks a lucrative target for theft. An attacker
can obtain a model without having access to the training data or incurring the
cost of training. Also, networks trained using private data (e.g., medical
records) can reveal information about this data. Networks can be stolen by
leveraging side channels such as power traces of the IoT device when it is
running the network. Existing attacks require operations to occur in the same
order each time; an attacker must collect and analyze several traces of the
device to steal the network. Therefore, to prevent this type of attack, we
randomly shuffle the order of operations each time. With shuffling, each
operation can now happen at many different points in each execution, making the
attack intractable. However, we show that shuffling in software can leak
information which can be used to subvert this solution. Therefore, to perform
secure shuffling and reduce latency, we present BlackJack, hardware added as a
functional unit within the CPU. BlackJack secures neural networks on IoT
devices by increasing the time needed for an attack to centuries, while adding
just 2.46% area, 3.28% power and 0.56% latency overhead on an ARM M0+ SoC.Comment: 16 pages, 6 figure
Defensive Approximation: Securing CNNs using Approximate Computing
In the past few years, an increasing number of machine-learning and deep
learning structures, such as Convolutional Neural Networks (CNNs), have been
applied to solving a wide range of real-life problems. However, these
architectures are vulnerable to adversarial attacks. In this paper, we propose
for the first time to use hardware-supported approximate computing to improve
the robustness of machine learning classifiers. We show that our approximate
computing implementation achieves robustness across a wide range of attack
scenarios. Specifically, for black-box and grey-box attack scenarios, we show
that successful adversarial attacks against the exact classifier have poor
transferability to the approximate implementation. Surprisingly, the robustness
advantages also apply to white-box attacks where the attacker has access to the
internal implementation of the approximate classifier. We explain some of the
possible reasons for this robustness through analysis of the internal operation
of the approximate implementation. Furthermore, our approximate computing model
maintains the same level in terms of classification accuracy, does not require
retraining, and reduces resource utilization and energy consumption of the CNN.
We conducted extensive experiments on a set of strong adversarial attacks; We
empirically show that the proposed implementation increases the robustness of a
LeNet-5 and an Alexnet CNNs by up to 99% and 87%, respectively for strong
grey-box adversarial attacks along with up to 67% saving in energy consumption
due to the simpler nature of the approximate logic. We also show that a
white-box attack requires a remarkably higher noise budget to fool the
approximate classifier, causing an average of 4db degradation of the PSNR of
the input image relative to the images that succeed in fooling the exact
classifierComment: ACM International Conference on Architectural Support for Programming
Languages and Operating Systems (ASPLOS 2021
Quantized Neural Networks and Neuromorphic Computing for Embedded Systems
Deep learning techniques have made great success in areas such as computer vision, speech recognition and natural language processing. Those breakthroughs made by deep learning techniques are changing every aspect of our lives. However, deep learning techniques have not realized their full potential in embedded systems such as mobiles, vehicles etc. because the high performance of deep learning techniques comes at the cost of high computation resource and energy consumption. Therefore, it is very challenging to deploy deep learning models in embedded systems because such systems have very limited computation resources and power constraints. Extensive research on deploying deep learning techniques in embedded systems has been conducted and considerable progress has been made. In this book chapter, we are going to introduce two approaches. The first approach is model compression, which is one of the very popular approaches proposed in recent years. Another approach is neuromorphic computing, which is a novel computing system that mimicks the human brain
Deep Complex Networks
At present, the vast majority of building blocks, techniques, and
architectures for deep learning are based on real-valued operations and
representations. However, recent work on recurrent neural networks and older
fundamental theoretical analysis suggests that complex numbers could have a
richer representational capacity and could also facilitate noise-robust memory
retrieval mechanisms. Despite their attractive properties and potential for
opening up entirely new neural architectures, complex-valued deep neural
networks have been marginalized due to the absence of the building blocks
required to design such models. In this work, we provide the key atomic
components for complex-valued deep neural networks and apply them to
convolutional feed-forward networks and convolutional LSTMs. More precisely, we
rely on complex convolutions and present algorithms for complex
batch-normalization, complex weight initialization strategies for
complex-valued neural nets and we use them in experiments with end-to-end
training schemes. We demonstrate that such complex-valued models are
competitive with their real-valued counterparts. We test deep complex models on
several computer vision tasks, on music transcription using the MusicNet
dataset and on Speech Spectrum Prediction using the TIMIT dataset. We achieve
state-of-the-art performance on these audio-related tasks
- …