5,549 research outputs found
Weightless: Lossy Weight Encoding For Deep Neural Network Compression
The large memory requirements of deep neural networks limit their deployment
and adoption on many devices. Model compression methods effectively reduce the
memory requirements of these models, usually through applying transformations
such as weight pruning or quantization. In this paper, we present a novel
scheme for lossy weight encoding which complements conventional compression
techniques. The encoding is based on the Bloomier filter, a probabilistic data
structure that can save space at the cost of introducing random errors.
Leveraging the ability of neural networks to tolerate these imperfections and
by re-training around the errors, the proposed technique, Weightless, can
compress DNN weights by up to 496x with the same model accuracy. This results
in up to a 1.51x improvement over the state-of-the-art
Distance Preserving Graph Simplification
Large graphs are difficult to represent, visualize, and understand. In this
paper, we introduce "gate graph" - a new approach to perform graph
simplification. A gate graph provides a simplified topological view of the
original graph. Specifically, we construct a gate graph from a large graph so
that for any "non-local" vertex pair (distance higher than some threshold) in
the original graph, their shortest-path distance can be recovered by
consecutive "local" walks through the gate vertices in the gate graph. We
perform a theoretical investigation on the gate-vertex set discovery problem.
We characterize its computational complexity and reveal the upper bound of
minimum gate-vertex set using VC-dimension theory. We propose an efficient
mining algorithm to discover a gate-vertex set with guaranteed logarithmic
bound. We further present a fast technique for pruning redundant edges in a
gate graph. The detailed experimental results using both real and synthetic
graphs demonstrate the effectiveness and efficiency of our approach.Comment: A short version of this paper will be published for ICDM'11, December
201
- …