92 research outputs found
Learning Sparse Neural Networks via Sensitivity-Driven Regularization
The ever-increasing number of parameters in deep neural networks poses
challenges for memory-limited applications. Regularize-and-prune methods aim at
meeting these challenges by sparsifying the network weights. In this context we
quantify the output sensitivity to the parameters (i.e. their relevance to the
network output) and introduce a regularization term that gradually lowers the
absolute value of parameters with low sensitivity. Thus, a very large fraction
of the parameters approach zero and are eventually set to zero by simple
thresholding. Our method surpasses most of the recent techniques both in terms
of sparsity and error rates. In some cases, the method reaches twice the
sparsity obtained by other techniques at equal error rates
Pollution-resilient peer-to-peer video streaming with Band Codes
Band Codes (BC) have been recently proposed as a solution for controlled-complexity random Network Coding (NC) in mobile applications, where energy consumption is a major concern. In this paper, we investigate the potential of BC in a peer-to-peer video streaming scenario where malicious and honest nodes coexists. Malicious nodes launch the so called pollution attack by randomly modifying the content of the coded packets they forward to downstream nodes, preventing honest nodes from correctly recovering the video stream. Whereas in much of the related literature this type of attack is addressed by identifying and isolating the malicious nodes, in this work we propose to address it by adaptively adjusting the coding scheme so to introduce resilience against pollution propagation. We experimentally show the impact of a pollution attack in a defenseless system and in a system where the coding parameters of BC are adaptively modulated following the discovery of polluted packets in the network. We observe that just by tuning the coding parameters, it is possible to reduce the impact of a pollution attack and restore the quality of the video communication
Characterization of Band Codes for Pollution-Resilient Peer-to-Peer Video Streaming
We provide a comprehensive characterization of band codes (BC) as a resilient-by-design solution to pollution attacks in network coding (NC)-based peer-to-peer live video streaming. Consider one malicious node injecting bogus coded packets into the network: the recombinations at the nodes generate an avalanche of novel coded bogus packets. Therefore, the malicious node can cripple the communication by injecting into the network only a handful of polluted packets. Pollution attacks are typically addressed by identifying and isolating the malicious nodes from the network. Pollution detection is, however, not straightforward in NC as the nodes exchange coded packets. Similarly, malicious nodes identification is complicated by the ambiguity between malicious nodes and nodes that have involuntarily relayed polluted packets. This paper addresses pollution attacks through a radically different approach which relies on BCs. BCs are a family of rateless codes originally designed for controlling the NC decoding complexity in mobile applications. Here, we exploit BCs for the totally different purpose of recombining the packets at the nodes so to avoid that the pollution propagates by adaptively adjusting the coding parameters. Our streaming experiments show that BCs curb the propagation of the pollution and restore the quality of the distributed video stream
GPU-Accelerated Algorithms for Compressed Signals Recovery with Application to Astronomical Imagery Deblurring
Compressive sensing promises to enable bandwidth-efficient on-board
compression of astronomical data by lifting the encoding complexity from the
source to the receiver. The signal is recovered off-line, exploiting GPUs
parallel computation capabilities to speedup the reconstruction process.
However, inherent GPU hardware constraints limit the size of the recoverable
signal and the speedup practically achievable. In this work, we design parallel
algorithms that exploit the properties of circulant matrices for efficient
GPU-accelerated sparse signals recovery. Our approach reduces the memory
requirements, allowing us to recover very large signals with limited memory. In
addition, it achieves a tenfold signal recovery speedup thanks to ad-hoc
parallelization of matrix-vector multiplications and matrix inversions.
Finally, we practically demonstrate our algorithms in a typical application of
circulant matrices: deblurring a sparse astronomical image in the compressed
domain
Band Codes for Energy-Efficient Network Coding with Application to P2P Mobile Streaming
A key problem in random network coding (NC) lies in the complexity and energy
consumption associated with the packet decoding processes, which hinder its
application in mobile environments. Controlling and hence limiting such factors
has always been an important but elusive research goal, since the packet degree
distribution, which is the main factor driving the complexity, is altered in a
non-deterministic way by the random recombinations at the network nodes. In
this paper we tackle this problem proposing Band Codes (BC), a novel class of
network codes specifically designed to preserve the packet degree distribution
during packet encoding, ecombination and decoding. BC are random codes over
GF(2) that exhibit low decoding complexity, feature limited and controlled
degree distribution by construction, and hence allow to effectively apply NC
even in energy-constrained scenarios. In particular, in this paper we motivate
and describe our new design and provide a thorough analysis of its performance.
We provide numerical simulations of the performance of BC in order to validate
the analysis and assess the overhead of BC with respect to a onventional NC
scheme. Moreover, peer-to-peer media streaming experiments with a random-push
protocol show that BC reduce the decoding complexity by a factor of two, to a
point where NC-based mobile streaming to mobile devices becomes practically
feasible.Comment: To be published in IEEE Transacions on Multimedi
LOss-Based SensiTivity rEgulaRization: Towards deep sparse neural networks
International audienc
LOss-Based SensiTivity rEgulaRization: towards deep sparse neural networks
LOBSTER (LOss-Based SensiTivity rEgulaRization) is a method for training
neural networks having a sparse topology. Let the sensitivity of a network
parameter be the variation of the loss function with respect to the variation
of the parameter. Parameters with low sensitivity, i.e. having little impact on
the loss when perturbed, are shrunk and then pruned to sparsify the network.
Our method allows to train a network from scratch, i.e. without preliminary
learning or rewinding. Experiments on multiple architectures and datasets show
competitive compression ratios with minimal computational overhead
- …