12 research outputs found
Supervised Learning Under Distributed Features
This work studies the problem of learning under both large datasets and
large-dimensional feature space scenarios. The feature information is assumed
to be spread across agents in a network, where each agent observes some of the
features. Through local cooperation, the agents are supposed to interact with
each other to solve an inference problem and converge towards the global
minimizer of an empirical risk. We study this problem exclusively in the primal
domain, and propose new and effective distributed solutions with guaranteed
convergence to the minimizer with linear rate under strong convexity. This is
achieved by combining a dynamic diffusion construction, a pipeline strategy,
and variance-reduced techniques. Simulation results illustrate the conclusions
Dynamic Average Diffusion with randomized Coordinate Updates
This work derives and analyzes an online learning strategy for tracking the
average of time-varying distributed signals by relying on randomized
coordinate-descent updates. During each iteration, each agent selects or
observes a random entry of the observation vector, and different agents may
select different entries of their observations before engaging in a
consultation step. Careful coordination of the interactions among agents is
necessary to avoid bias and ensure convergence. We provide a convergence
analysis for the proposed methods, and illustrate the results by means of
simulations
Decentralized Non-Convex Learning with Linearly Coupled Constraints
Motivated by the need for decentralized learning, this paper aims at
designing a distributed algorithm for solving nonconvex problems with general
linear constraints over a multi-agent network. In the considered problem, each
agent owns some local information and a local variable for jointly minimizing a
cost function, but local variables are coupled by linear constraints. Most of
the existing methods for such problems are only applicable for convex problems
or problems with specific linear constraints. There still lacks a distributed
algorithm for such problems with general linear constraints and under nonconvex
setting. In this paper, to tackle this problem, we propose a new algorithm,
called "proximal dual consensus" (PDC) algorithm, which combines a proximal
technique and a dual consensus method. We build the theoretical convergence
conditions and show that the proposed PDC algorithm can converge to an
-Karush-Kuhn-Tucker solution within
iterations. For computation reduction, the PDC algorithm can choose to perform
cheap gradient descent per iteration while preserving the same order of
iteration complexity. Numerical results are presented
to demonstrate the good performance of the proposed algorithms for solving a
regression problem and a classification problem over a network where agents
have only partial observations of data features