57,298 research outputs found
Communication and Memory Efficient Testing of Discrete Distributions
We study distribution testing with communication and memory constraints in
the following computational models: (1) The {\em one-pass streaming model}
where the goal is to minimize the sample complexity of the protocol subject to
a memory constraint, and (2) A {\em distributed model} where the data samples
reside at multiple machines and the goal is to minimize the communication cost
of the protocol. In both these models, we provide efficient algorithms for
uniformity/identity testing (goodness of fit) and closeness testing (two sample
testing). Moreover, we show nearly-tight lower bounds on (1) the sample
complexity of any one-pass streaming tester for uniformity, subject to the
memory constraint, and (2) the communication cost of any uniformity testing
protocol, in a restricted `one-pass' model of communication.Comment: Full version of COLT 2019 pape
Construction and Verification of Performance and Reliability Models
Over the last two decades formal methods have been extended towards performance and reliability evaluation. This paper tries to provide a rather intuitive explanation of the basic concepts and features in this area.
Instead of striving for mathematical rigour, the intention is to give an illustrative introduction to the basics of stochastic models, to stochastic modelling using process algebra, and to model checking as a technique to analyse stochastic models
Inherent Weight Normalization in Stochastic Neural Networks
Multiplicative stochasticity such as Dropout improves the robustness and
generalizability of deep neural networks. Here, we further demonstrate that
always-on multiplicative stochasticity combined with simple threshold neurons
are sufficient operations for deep neural networks. We call such models Neural
Sampling Machines (NSM). We find that the probability of activation of the NSM
exhibits a self-normalizing property that mirrors Weight Normalization, a
previously studied mechanism that fulfills many of the features of Batch
Normalization in an online fashion. The normalization of activities during
training speeds up convergence by preventing internal covariate shift caused by
changes in the input distribution. The always-on stochasticity of the NSM
confers the following advantages: the network is identical in the inference and
learning phases, making the NSM suitable for online learning, it can exploit
stochasticity inherent to a physical substrate such as analog non-volatile
memories for in-memory computing, and it is suitable for Monte Carlo sampling,
while requiring almost exclusively addition and comparison operations. We
demonstrate NSMs on standard classification benchmarks (MNIST and CIFAR) and
event-based classification benchmarks (N-MNIST and DVS Gestures). Our results
show that NSMs perform comparably or better than conventional artificial neural
networks with the same architecture
- ā¦