2,985 research outputs found
NOMAD: Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion
We develop an efficient parallel distributed algorithm for matrix completion,
named NOMAD (Non-locking, stOchastic Multi-machine algorithm for Asynchronous
and Decentralized matrix completion). NOMAD is a decentralized algorithm with
non-blocking communication between processors. One of the key features of NOMAD
is that the ownership of a variable is asynchronously transferred between
processors in a decentralized fashion. As a consequence it is a lock-free
parallel algorithm. In spite of being an asynchronous algorithm, the variable
updates of NOMAD are serializable, that is, there is an equivalent update
ordering in a serial implementation. NOMAD outperforms synchronous algorithms
which require explicit bulk synchronization after every iteration: our
extensive empirical evaluation shows that not only does our algorithm perform
well in distributed setting on commodity hardware, but also outperforms
state-of-the-art algorithms on a HPC cluster both in multi-core and distributed
memory settings
Adaptation and learning over networks for nonlinear system modeling
In this chapter, we analyze nonlinear filtering problems in distributed
environments, e.g., sensor networks or peer-to-peer protocols. In these
scenarios, the agents in the environment receive measurements in a streaming
fashion, and they are required to estimate a common (nonlinear) model by
alternating local computations and communications with their neighbors. We
focus on the important distinction between single-task problems, where the
underlying model is common to all agents, and multitask problems, where each
agent might converge to a different model due to, e.g., spatial dependencies or
other factors. Currently, most of the literature on distributed learning in the
nonlinear case has focused on the single-task case, which may be a strong
limitation in real-world scenarios. After introducing the problem and reviewing
the existing approaches, we describe a simple kernel-based algorithm tailored
for the multitask case. We evaluate the proposal on a simulated benchmark task,
and we conclude by detailing currently open problems and lines of research.Comment: To be published as a chapter in `Adaptive Learning Methods for
Nonlinear System Modeling', Elsevier Publishing, Eds. D. Comminiello and J.C.
Principe (2018
- …