26 research outputs found
NOMAD: Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion
We develop an efficient parallel distributed algorithm for matrix completion,
named NOMAD (Non-locking, stOchastic Multi-machine algorithm for Asynchronous
and Decentralized matrix completion). NOMAD is a decentralized algorithm with
non-blocking communication between processors. One of the key features of NOMAD
is that the ownership of a variable is asynchronously transferred between
processors in a decentralized fashion. As a consequence it is a lock-free
parallel algorithm. In spite of being an asynchronous algorithm, the variable
updates of NOMAD are serializable, that is, there is an equivalent update
ordering in a serial implementation. NOMAD outperforms synchronous algorithms
which require explicit bulk synchronization after every iteration: our
extensive empirical evaluation shows that not only does our algorithm perform
well in distributed setting on commodity hardware, but also outperforms
state-of-the-art algorithms on a HPC cluster both in multi-core and distributed
memory settings