1,269 research outputs found
Distributed Machine Learning via Sufficient Factor Broadcasting
Matrix-parametrized models, including multiclass logistic regression and
sparse coding, are used in machine learning (ML) applications ranging from
computer vision to computational biology. When these models are applied to
large-scale ML problems starting at millions of samples and tens of thousands
of classes, their parameter matrix can grow at an unexpected rate, resulting in
high parameter synchronization costs that greatly slow down distributed
learning. To address this issue, we propose a Sufficient Factor Broadcasting
(SFB) computation model for efficient distributed learning of a large family of
matrix-parameterized models, which share the following property: the parameter
update computed on each data sample is a rank-1 matrix, i.e., the outer product
of two "sufficient factors" (SFs). By broadcasting the SFs among worker
machines and reconstructing the update matrices locally at each worker, SFB
improves communication efficiency --- communication costs are linear in the
parameter matrix's dimensions, rather than quadratic --- without affecting
computational correctness. We present a theoretical convergence analysis of
SFB, and empirically corroborate its efficiency on four different
matrix-parametrized ML models
Agent-Based Simulations of Blockchain protocols illustrated via Kadena's Chainweb
While many distributed consensus protocols provide robust liveness and
consistency guarantees under the presence of malicious actors, quantitative
estimates of how economic incentives affect security are few and far between.
In this paper, we describe a system for simulating how adversarial agents, both
economically rational and Byzantine, interact with a blockchain protocol. This
system provides statistical estimates for the economic difficulty of an attack
and how the presence of certain actors influences protocol-level statistics,
such as the expected time to regain liveness. This simulation system is
influenced by the design of algorithmic trading and reinforcement learning
systems that use explicit modeling of an agent's reward mechanism to evaluate
and optimize a fully autonomous agent. We implement and apply this simulation
framework to Kadena's Chainweb, a parallelized Proof-of-Work system, that
contains complexity in how miner incentive compliance affects security and
censorship resistance. We provide the first formal description of Chainweb that
is in the literature and use this formal description to motivate our simulation
design. Our simulation results include a phase transition in block height
growth rate as a function of shard connectivity and empirical evidence that
censorship in Chainweb is too costly for rational miners to engage in. We
conclude with an outlook on how simulation can guide and optimize protocol
development in a variety of contexts, including Proof-of-Stake parameter
optimization and peer-to-peer networking design.Comment: 10 pages, 7 figures, accepted to the IEEE S&B 2019 conferenc
- …