261,113 research outputs found
Optimal Rates for Distributed Learning with Random Features
In recent studies, the generalization properties for distributed learning and
random features assumed the existence of the target concept over the hypothesis
space. However, this strict condition is not applicable to the more common
non-attainable case. In this paper, using refined proof techniques, we first
extend the optimal rates for distributed learning with random features to the
non-attainable case. Then, we reduce the number of required random features via
data-dependent generating strategy, and improve the allowed number of
partitions with additional unlabeled data. Theoretical analysis shows these
techniques remarkably reduce computational cost while preserving the optimal
generalization accuracy under standard assumptions. Finally, we conduct several
experiments on both simulated and real-world datasets, and the empirical
results validate our theoretical findings.Comment: Accpected at IJCAI 202
Learning from distributed data sources using random vector functional-link networks
One of the main characteristics in many real-world big data scenarios is their distributed nature. In a machine learning context, distributed data, together with the requirements of preserving privacy and scaling up to large networks, brings the challenge of designing fully decentralized training protocols. In this paper, we explore the problem of distributed learning when the features of every pattern are available throughout multiple agents (as is happening, for example, in a distributed database scenario). We propose an algorithm for a particular class of neural networks, known as Random Vector Functional-Link (RVFL), which is based on the Alternating Direction Method of Multipliers optimization algorithm. The proposed algorithm allows to learn an RVFL network from multiple distributed data sources, while restricting communication to the unique operation of computing a distributed average. Our experimental simulations show that the algorithm is able to achieve a generalization accuracy comparable to a fully centralized solution, while at the same time being extremely efficient
- …