5,321 research outputs found
On the Convergence of Decentralized Gradient Descent
Consider the consensus problem of minimizing where
each is only known to one individual agent out of a connected network
of agents. All the agents shall collaboratively solve this problem and
obtain the solution subject to data exchanges restricted to between neighboring
agents. Such algorithms avoid the need of a fusion center, offer better network
load balance, and improve data privacy. We study the decentralized gradient
descent method in which each agent updates its variable , which is
a local approximate to the unknown variable , by combining the average of
its neighbors' with the negative gradient step .
The iteration is where the averaging coefficients form a symmetric doubly stochastic matrix
. We analyze the convergence of this
iteration and derive its converge rate, assuming that each is proper
closed convex and lower bounded, is Lipschitz continuous with
constant , and stepsize is fixed. Provided that where , the objective error at the averaged
solution, , reduces at a speed of
until it reaches . If are further (restricted) strongly
convex, then both and each converge
to the global minimizer at a linear rate until reaching an
-neighborhood of . We also develop an iteration for
decentralized basis pursuit and establish its linear convergence to an
-neighborhood of the true unknown sparse signal
Supervised Learning Under Distributed Features
This work studies the problem of learning under both large datasets and
large-dimensional feature space scenarios. The feature information is assumed
to be spread across agents in a network, where each agent observes some of the
features. Through local cooperation, the agents are supposed to interact with
each other to solve an inference problem and converge towards the global
minimizer of an empirical risk. We study this problem exclusively in the primal
domain, and propose new and effective distributed solutions with guaranteed
convergence to the minimizer with linear rate under strong convexity. This is
achieved by combining a dynamic diffusion construction, a pipeline strategy,
and variance-reduced techniques. Simulation results illustrate the conclusions
Towards High-Fidelity 3D Face Reconstruction from In-the-Wild Images Using Graph Convolutional Networks
3D Morphable Model (3DMM) based methods have achieved great success in
recovering 3D face shapes from single-view images. However, the facial textures
recovered by such methods lack the fidelity as exhibited in the input images.
Recent work demonstrates high-quality facial texture recovering with generative
networks trained from a large-scale database of high-resolution UV maps of face
textures, which is hard to prepare and not publicly available. In this paper,
we introduce a method to reconstruct 3D facial shapes with high-fidelity
textures from single-view images in-the-wild, without the need to capture a
large-scale face texture database. The main idea is to refine the initial
texture generated by a 3DMM based method with facial details from the input
image. To this end, we propose to use graph convolutional networks to
reconstruct the detailed colors for the mesh vertices instead of reconstructing
the UV map. Experiments show that our method can generate high-quality results
and outperforms state-of-the-art methods in both qualitative and quantitative
comparisons.Comment: Accepted to CVPR 2020. The source code is available at
https://github.com/FuxiCV/3D-Face-GCN
Photoproduction of in NRQCD
We present a calculation for the photoproduction of under the
framework of NRQCD factorization formalism. We find a quite unique feature that
the color-singlet contribution to this process vanishes at not only the leading
order but also the next to leading order perturbative QCD calculations and that
the dominant contribution comes from the color-octet
subprocess. The nonperturbative color-octet matrix element of
of is related to that of of by the heavy
quark spin symmetry, and the latter can be determined from the direct
production of at large transverse momentum at the Fermilib Tevatron.
We then conclude that the measurement of this process may clarify the existing
conflict between the color-octet prediction and the experimental result on the
photoprodution.Comment: 11 pages, revtex, 4 ps figure
Variance-Reduced Stochastic Learning by Networked Agents under Random Reshuffling
A new amortized variance-reduced gradient (AVRG) algorithm was developed in
\cite{ying2017convergence}, which has constant storage requirement in
comparison to SAGA and balanced gradient computations in comparison to SVRG.
One key advantage of the AVRG strategy is its amenability to decentralized
implementations. In this work, we show how AVRG can be extended to the network
case where multiple learning agents are assumed to be connected by a graph
topology. In this scenario, each agent observes data that is spatially
distributed and all agents are only allowed to communicate with direct
neighbors. Moreover, the amount of data observed by the individual agents may
differ drastically. For such situations, the balanced gradient computation
property of AVRG becomes a real advantage in reducing idle time caused by
unbalanced local data storage requirements, which is characteristic of other
reduced-variance gradient algorithms. The resulting diffusion-AVRG algorithm is
shown to have linear convergence to the exact solution, and is much more memory
efficient than other alternative algorithms. In addition, we propose a
mini-batch strategy to balance the communication and computation efficiency for
diffusion-AVRG. When a proper batch size is employed, it is observed in
simulations that diffusion-AVRG is more computationally efficient than exact
diffusion or EXTRA while maintaining almost the same communication efficiency.Comment: 23 pages, 12 figures, submitted for publicatio
- …