Expander Graph and Communication-Efficient Decentralized Optimization

In this paper, we discuss how to design the graph topology to reduce the communication complexity of certain algorithms for decentralized optimization. Our goal is to minimize the total communication needed to achieve a prescribed accuracy. We discover that the so-called expander graphs are near-optimal choices. We propose three approaches to construct expander graphs for different numbers of nodes and node degrees. Our numerical results show that the performance of decentralized optimization is significantly better on expander graphs than other regular graphs.Comment: 2016 IEEE Asilomar Conference on Signals, Systems, and Computer

Decentralized projected Riemannian gradient method for smooth optimization on compact submanifolds

We consider the problem of decentralized nonconvex optimization over a compact submanifold, where each local agent's objective function defined by the local dataset is smooth. Leveraging the powerful tool of proximal smoothness, we establish local linear convergence of the projected gradient descent method with unit step size for solving the consensus problem over the compact manifold. This serves as the basis for analyzing decentralized algorithms on manifolds. Then, we propose two decentralized methods, namely the decentralized projected Riemannian gradient descent (DPRGD) and the decentralized projected Riemannian gradient tracking (DPRGT) methods. We establish their convergence rates of $\mathcal{O}(1/\sqrt{K})$ and $\mathcal{O}(1/K)$, respectively, to reach a stationary point. To the best of our knowledge, DPRGT is the first decentralized algorithm to achieve exact convergence for solving decentralized optimization over a compact manifold. The key ingredients in the proof are the Lipschitz-type inequalities of the projection operator on the compact manifold and smooth functions on the manifold, which could be of independent interest. Finally, we demonstrate the effectiveness of our proposed methods compared to state-of-the-art ones through numerical experiments on eigenvalue problems and low-rank matrix completion.Comment: 32 page

Asynchronous Distributed ADMM for Large-Scale Optimization- Part I: Algorithm and Convergence Analysis

Aiming at solving large-scale learning problems, this paper studies distributed optimization methods based on the alternating direction method of multipliers (ADMM). By formulating the learning problem as a consensus problem, the ADMM can be used to solve the consensus problem in a fully parallel fashion over a computer network with a star topology. However, traditional synchronized computation does not scale well with the problem size, as the speed of the algorithm is limited by the slowest workers. This is particularly true in a heterogeneous network where the computing nodes experience different computation and communication delays. In this paper, we propose an asynchronous distributed ADMM (AD-AMM) which can effectively improve the time efficiency of distributed optimization. Our main interest lies in analyzing the convergence conditions of the AD-ADMM, under the popular partially asynchronous model, which is defined based on a maximum tolerable delay of the network. Specifically, by considering general and possibly non-convex cost functions, we show that the AD-ADMM is guaranteed to converge to the set of Karush-Kuhn-Tucker (KKT) points as long as the algorithm parameters are chosen appropriately according to the network delay. We further illustrate that the asynchrony of the ADMM has to be handled with care, as slightly modifying the implementation of the AD-ADMM can jeopardize the algorithm convergence, even under a standard convex setting.Comment: 37 page

Fully Distributed Robust Singular Value Decomposition

Articulated Funiculator is a new and innovative concept developed by Tyréns forachieving a more efficient vertical transportation with a higher space utilization.Having a variety of merits, i.e.: simple construction, direct electromagneticthrust propulsion, and high safety and reliability in contrast to rotary inductionmotor, linear induction motor (LIM) is considered to be one of the cases as thepropulsion system for Articulated Funiculator. The thesis is then carried outwith the purpose of determining the feasibility of this study case by designing theLIMs meeting some specific requirements. The detailed requirements include: aset of identical LIMs are required to jointly produce the thrust that is sufficientto vertically raise the moving system up to 2 m/s2; the size of the LIMs cannotexceed the specification of the funiculator; the maximum flux density in the airgap for each LIM is kept slightly below 0.6 T; no iron saturation of any part ofthe LIMs is allowed.In this thesis report, an introduction of LIM is firstly presented. Followingthe introduction, relevant literature has been reviewed for a strengthenedtheoretical fundamentals and a better understanding of LIM’s history and applications. A general classification of LIMs is subsequently introduced. In addtion,an analytical model of the single-sided linear induction motor (SLIM) has beenbuilt based on an approximate equivalent circuit, and the preliminary geometryof the SLIM is thereby obtained. In order to acquire a more comprehensiveunderstanding of the machine characteristics and a more precise SLIM design, atwo-dimensional finite element method (2D-FEM) analysis is performed initiallyaccording to the preliminary geometry. The results, unfortunately, turn out tobe iron severely saturated in the teeth and yoke, and a excessive maximumvalue of air-gap flux density. Specific to the problems, different parameters ofthe SLIM are marginally adjusted and a series of design scenarios are run inFlux2D for 8-pole and 6-pole SLIM. The comparisons between the results areconducted and the final solution is lastly chosen among them.Articulated Funiculator är ett nytt och innovativt koncept som utvecklats av Tyréns för att möjilggöra en mer effektiv vertikal transport och bättre utnyttjautrymme. Tack vare fördelar såsom en enkel konstruktion, direkt elektromagnetiskdragkraftsframdrivning, samt hög säkerhet och tillförlitlighet i motsatstill roterande induktionsmotor, är en linjär induktionsmotor (LIM) aktuell somframdrivningssystem. Detta examensarbete är utfört med syfte att utforma enLIM för att uppfylla vissa specifika krav. De detaljerade kraven inkluderar: enuppsättning identiska LIM krävs för att gemensamt producera tillräcklig dragkraftför att vertikalt höja det rörliga systemet upp till 2 m/s2; storleken påLIM får inte överstiga specifikation; den maximala flödestätheten i luftgapet förvarje LIM hålls är begränsad till knappt 0.6 T; ingen järnmättnad av någon delav LIM är tillåtet. I denna rapport ges först en introduktion av LIM-konceptet. Efter introduktionenhar relevant litteratur granskats för att stärka teoretiska grundkunskapersamt ge en bättre belysning av historiken kring LIMs samt dess applikationer. Utöver detta har en analytisk modell av den ensidiga linjära induktionsmotorn(SLIM) byggts, baserat på en ungefärlig ekvivalent krets med vilket den preliminärageometrin för SLIM. För att erhålla en mer grundläggande förståelse avmaskinens egenskaper är en tvådimensionell analys med finita elementmetoden(2D-FEM) utförd, initialt med användande av en preliminär geometri erhållenmed hjälp av analytisk dimensionering. Resultaten från dessa simuleringar visadedock att järnet mättats kraftigt i både tänderna och oket och ett överdrivetstort maximivärde av luftgapets flödestäthet erhålls. Specifikt för applikationenjusteras olika parametrar och en rad driftscenarier körs i Flux2D för en 8-poligoch en 6-polig SLIM. En slutgiltig jämförelse mellan de olika maskindesignernapresenteras och den rekommenderade lösningen väljs slutligen