57,585 research outputs found
Balanced Sparsity for Efficient DNN Inference on GPU
In trained deep neural networks, unstructured pruning can reduce redundant
weights to lower storage cost. However, it requires the customization of
hardwares to speed up practical inference. Another trend accelerates sparse
model inference on general-purpose hardwares by adopting coarse-grained
sparsity to prune or regularize consecutive weights for efficient computation.
But this method often sacrifices model accuracy. In this paper, we propose a
novel fine-grained sparsity approach, balanced sparsity, to achieve high model
accuracy with commercial hardwares efficiently. Our approach adapts to high
parallelism property of GPU, showing incredible potential for sparsity in the
widely deployment of deep learning services. Experiment results show that
balanced sparsity achieves up to 3.1x practical speedup for model inference on
GPU, while retains the same high model accuracy as fine-grained sparsity
Feedback-prop: Convolutional Neural Network Inference under Partial Evidence
We propose an inference procedure for deep convolutional neural networks
(CNNs) when partial evidence is available. Our method consists of a general
feedback-based propagation approach (feedback-prop) that boosts the prediction
accuracy for an arbitrary set of unknown target labels when the values for a
non-overlapping arbitrary set of target labels are known. We show that existing
models trained in a multi-label or multi-task setting can readily take
advantage of feedback-prop without any retraining or fine-tuning. Our
feedback-prop inference procedure is general, simple, reliable, and works on
different challenging visual recognition tasks. We present two variants of
feedback-prop based on layer-wise and residual iterative updates. We experiment
using several multi-task models and show that feedback-prop is effective in all
of them. Our results unveil a previously unreported but interesting dynamic
property of deep CNNs. We also present an associated technical approach that
takes advantage of this property for inference under partial evidence in
general visual recognition tasks.Comment: Accepted to CVPR 201
Inherent Weight Normalization in Stochastic Neural Networks
Multiplicative stochasticity such as Dropout improves the robustness and
generalizability of deep neural networks. Here, we further demonstrate that
always-on multiplicative stochasticity combined with simple threshold neurons
are sufficient operations for deep neural networks. We call such models Neural
Sampling Machines (NSM). We find that the probability of activation of the NSM
exhibits a self-normalizing property that mirrors Weight Normalization, a
previously studied mechanism that fulfills many of the features of Batch
Normalization in an online fashion. The normalization of activities during
training speeds up convergence by preventing internal covariate shift caused by
changes in the input distribution. The always-on stochasticity of the NSM
confers the following advantages: the network is identical in the inference and
learning phases, making the NSM suitable for online learning, it can exploit
stochasticity inherent to a physical substrate such as analog non-volatile
memories for in-memory computing, and it is suitable for Monte Carlo sampling,
while requiring almost exclusively addition and comparison operations. We
demonstrate NSMs on standard classification benchmarks (MNIST and CIFAR) and
event-based classification benchmarks (N-MNIST and DVS Gestures). Our results
show that NSMs perform comparably or better than conventional artificial neural
networks with the same architecture
Efficient Approximate Inference with Walsh-Hadamard Variational Inference
Variational inference offers scalable and flexible tools to tackle
intractable Bayesian inference of modern statistical models like Bayesian
neural networks and Gaussian processes. For largely over-parameterized models,
however, the over-regularization property of the variational objective makes
the application of variational inference challenging. Inspired by the
literature on kernel methods, and in particular on structured approximations of
distributions of random matrices, this paper proposes Walsh-Hadamard
Variational Inference, which uses Walsh-Hadamard-based factorization strategies
to reduce model parameterization, accelerate computations, and increase the
expressiveness of the approximate posterior beyond fully factorized ones.Comment: Paper accepted at the 4th Workshop on Bayesian Deep Learning (NeurIPS
2019), Vancouver, Canada. arXiv admin note: substantial text overlap with
arXiv:1905.1124
SNAP: Efficient Extraction of Private Properties with Poisoning
Property inference attacks allow an adversary to extract global properties of
the training dataset from a machine learning model. Such attacks have privacy
implications for data owners sharing their datasets to train machine learning
models. Several existing approaches for property inference attacks against deep
neural networks have been proposed, but they all rely on the attacker training
a large number of shadow models, which induces a large computational overhead.
In this paper, we consider the setting of property inference attacks in which
the attacker can poison a subset of the training dataset and query the trained
target model. Motivated by our theoretical analysis of model confidences under
poisoning, we design an efficient property inference attack, SNAP, which
obtains higher attack success and requires lower amounts of poisoning than the
state-of-the-art poisoning-based property inference attack by Mahloujifar et
al. For example, on the Census dataset, SNAP achieves 34% higher success rate
than Mahloujifar et al. while being 56.5x faster. We also extend our attack to
infer whether a certain property was present at all during training and
estimate the exact proportion of a property of interest efficiently. We
evaluate our attack on several properties of varying proportions from four
datasets and demonstrate SNAP's generality and effectiveness. An open-source
implementation of SNAP can be found at https://github.com/johnmath/snap-sp23.Comment: 28 pages, 16 figure
- …