7,035 research outputs found
Scalable and Sustainable Deep Learning via Randomized Hashing
Current deep learning architectures are growing larger in order to learn from
complex datasets. These architectures require giant matrix multiplication
operations to train millions of parameters. Conversely, there is another
growing trend to bring deep learning to low-power, embedded devices. The matrix
operations, associated with both training and testing of deep networks, are
very expensive from a computational and energy standpoint. We present a novel
hashing based technique to drastically reduce the amount of computation needed
to train and test deep networks. Our approach combines recent ideas from
adaptive dropouts and randomized hashing for maximum inner product search to
select the nodes with the highest activation efficiently. Our new algorithm for
deep learning reduces the overall computational cost of forward and
back-propagation by operating on significantly fewer (sparse) nodes. As a
consequence, our algorithm uses only 5% of the total multiplications, while
keeping on average within 1% of the accuracy of the original model. A unique
property of the proposed hashing based back-propagation is that the updates are
always sparse. Due to the sparse gradient updates, our algorithm is ideally
suited for asynchronous and parallel training leading to near linear speedup
with increasing number of cores. We demonstrate the scalability and
sustainability (energy efficiency) of our proposed algorithm via rigorous
experimental evaluations on several real datasets
Multiparticle entanglement purification for two-colorable graph states
We investigate multiparticle entanglement purification schemes which allow
one to purify all two colorable graph states, a class of states which includes
e.g. cluster states, GHZ states and codewords of various error correction
codes. The schemes include both recurrence protocols and hashing protocols. We
analyze these schemes under realistic conditions and observe for a generic
error model that the threshold value for imperfect local operations depends on
the structure of the corresponding interaction graph, but is otherwise
independent of the number of parties. The qualitative behavior can be
understood from an analytically solvable model which deals only with a
restricted class of errors. We compare direct multiparticle entanglement
purification protocols with schemes based on bipartite entanglement
purification and show that the direct multiparticle entanglement purification
is more efficient and the achievable fidelity of the purified states is larger.
We also show that the purification protocol allows one to produce private
entanglement, an important aspect when using the produced entangled states for
secure applications. Finally we discuss an experimental realization of a
multiparty purification protocol in optical lattices which is issued to improve
the fidelity of cluster states created in such systems.Comment: 22 pages, 8 figures; replaced with published versio
Fast Scalable Construction of (Minimal Perfect Hash) Functions
Recent advances in random linear systems on finite fields have paved the way
for the construction of constant-time data structures representing static
functions and minimal perfect hash functions using less space with respect to
existing techniques. The main obstruction for any practical application of
these results is the cubic-time Gaussian elimination required to solve these
linear systems: despite they can be made very small, the computation is still
too slow to be feasible.
In this paper we describe in detail a number of heuristics and programming
techniques to speed up the resolution of these systems by several orders of
magnitude, making the overall construction competitive with the standard and
widely used MWHC technique, which is based on hypergraph peeling. In
particular, we introduce broadword programming techniques for fast equation
manipulation and a lazy Gaussian elimination algorithm. We also describe a
number of technical improvements to the data structure which further reduce
space usage and improve lookup speed.
Our implementation of these techniques yields a minimal perfect hash function
data structure occupying 2.24 bits per element, compared to 2.68 for MWHC-based
ones, and a static function data structure which reduces the multiplicative
overhead from 1.23 to 1.03
- …