A Convergence Result for Asynchronous Algorithms and Applications

We give in this paper a convergence result concerning parallel asynchronous algorithm with bounded delays to solve a nonlinear fixed point problems. This result is applied to calculate the solution of a strongly monotone operator. Special cases of these operators are used to solve some problems related to convex analysis like minimization of functionals, calculus of saddle point and variational inequality problem

On the Convergence Analysis of Asynchronous Distributed Quadratic Programming via Dual Decomposition

In this paper, we analyze the convergence as well as the rate of convergence of asynchronous distributed quadratic programming (QP) with dual decomposition technique. In general, distributed optimization requires synchronization of data at each iteration step due to the interdependency of data. This synchronization latency may incur a large amount of waiting time caused by an idle process during computation. We aim to attack this synchronization penalty in distributed QP problems by implementing asynchronous update of dual variable. The price to pay for adopting asynchronous computing algorithms is unpredictability of the solution, resulting in a tradeoff between speedup and accuracy. Thus, the convergence to an optimal solution is not guaranteed owing to the stochastic behavior of asynchrony. In this paper, we employ the switched system framework as an analysis tool to investigate the convergence of asynchronous distributed QP. This switched system will facilitate analysis on asynchronous distributed QP with dual decomposition, providing necessary and sufficient conditions for the mean square convergence. Also, we provide an analytic expression for the rate of convergence through the switched system, which enables performance analysis of asynchronous algorithms as compared with synchronous case. To verify the validity of the proposed methods, numerical examples are presented with an implementation of asynchronous parallel QP using OpenMP

Scheduled-Asynchronous Distributed Algorithm for Optimal Power Flow

Optimal power flow (OPF) problems are non-convex and large-scale optimization problems with important applications in power networks. This paper proposes the scheduled-asynchronous algorithm to solve a distributed semidefinite programming (SDP) formulation of the OPF problem. In this formulation, every agent seeks to solve a local optimization with its own cost function, physical constraints on its nodal power injection, voltage, and power flow of the lines it is connected to, and decision constraints on variables shared with neighbors to ensure consistency of the obtained solution. In the scheduled-asynchronous algorithm, every pair of connected nodes in the electrical network update their local variables in an alternating fashion. This strategy is asynchronous, in the sense that no clock synchronization is required, and relies on an orientation of the electrical network that prescribes the precise ordering of node updates. We establish the asymptotic convergence properties to the primal-dual optimizer when the orientation is acyclic. Given the dependence of the convergence rate on the network orientation, we also develop a distributed graph coloring algorithm that finds an orientation with diameter at most five for electrical networks with geometric degree distribution. Simulations illustrate our results on various IEEE bus test cases

Generalizing Parallel Replica Dynamics: Trajectory Fragments, Asynchronous Computing, and PDMPs

We study the Parallel Replica Dynamics in a general setting. We introduce a trajectory fragment framework that can be used to design and prove consistency of Parallel Replica algorithms for generic Markov processes. We use our framework to formulate a novel condition that guarantees an asynchronous algorithm is consistent. Exploiting this condition and our trajectory fragment framework, we present new synchronous and asynchronous Parallel Replica algorithms for piecewise deterministic Markov processes.Comment: 32 pages, 9 figure

Asynchronous Distributed Power Control of Multi-Microgrid Systems Based on the Operator Splitting Approach

Forming (hybrid) AC/DC microgrids (MGs) has become a promising manner for the interconnection of various kinds of distributed generators that are inherently AC or DC electric sources. This paper addresses the distributed asynchronous power control problem of hybrid microgrids, considering imperfect communication due to non-identical sampling rates and communication delays. To this end, we first formulate the optimal power control problem of MGs and devise a synchronous algorithm. Then, we analyze the impact of asynchrony on optimal power control and propose an asynchronous iteration algorithm based on the synchronous version. By introducing a random clock at each iteration, different types of asynchrony are fitted into a unified framework, where the asynchronous algorithm is converted into a fixed-point problem based on the operator splitting method, leading to a convergence proof. We further provide an upper bound estimation of the time delay in the communication. Moreover, the real-time implementation of the proposed algorithm in both AC and DC MGs is introduced. By taking the power system as a solver, the controller is simplified by reducing one order and the power loss can be considered. Finally, a benchmark MG is utilized to verify the effectiveness and advantages of the proposed algorithm

Methods of robustness analysis for Boolean models of gene control networks

As a discrete approach to genetic regulatory networks, Boolean models provide an essential qualitative description of the structure of interactions among genes and proteins. Boolean models generally assume only two possible states (expressed or not expressed) for each gene or protein in the network as well as a high level of synchronization among the various regulatory processes. In this paper, we discuss and compare two possible methods of adapting qualitative models to incorporate the continuous-time character of regulatory networks. The first method consists of introducing asynchronous updates in the Boolean model. In the second method, we adopt the approach introduced by L. Glass to obtain a set of piecewise linear differential equations which continuously describe the states of each gene or protein in the network. We apply both methods to a particular example: a Boolean model of the segment polarity gene network of Drosophila melanogaster. We analyze the dynamics of the model, and provide a theoretical characterization of the model's gene pattern prediction as a function of the timescales of the various processes.Comment: 29 pages, 8 figures, accepted in IEE Proc. Systems Biolog

Asynchronous Algorithms for Solving Linear Programs

In this paper we design and analyze algorithms for asynchronously solving linear programs using nonlinear signal processing structures. In particular, we discuss a general procedure for generating these structures such that a fixed-point of the structure is within a change of basis the minimizer of an associated linear program. We discuss methods for organizing the computation into distributed implementations and provide a treatment of convergence. The presented algorithms are accompanied by numerical simulations of the Chebyshev center and basis pursuit problems

Asynchronous Optimization Over Heterogeneous Networks via Consensus ADMM

This paper considers the distributed optimization of a sum of locally observable, non-convex functions. The optimization is performed over a multi-agent networked system, and each local function depends only on a subset of the variables. An asynchronous and distributed alternating directions method of multipliers (ADMM) method that allows the nodes to defer or skip the computation and transmission of updates is proposed in the paper. The proposed algorithm utilizes different approximations in the update step, resulting in proximal and majorized ADMM variants. Both variants are shown to converge to a local minimum, under certain regularity conditions. The proposed asynchronous algorithms are also applied to the problem of cooperative localization in wireless ad hoc networks, where it is shown to outperform the other state-of-the-art localization algorithms.Comment: Submitted to Transactions on signal and information processing over Network

A Switched Dynamical System Framework for Analysis of Massively Parallel Asynchronous Numerical Algorithms

In the near future, massively parallel computing systems will be necessary to solve computation intensive applications. The key bottleneck in massively parallel implementation of numerical algorithms is the synchronization of data across processing elements (PEs) after each iteration, which results in significant idle time. Thus, there is a trend towards relaxing the synchronization and adopting an asynchronous model of computation to reduce idle time. However, it is not clear what is the effect of this relaxation on the stability and accuracy of the numerical algorithm. In this paper we present a new framework to analyze such algorithms. We treat the computation in each PE as a dynamical system and model the asynchrony as stochastic switching. The overall system is then analyzed as a switched dynamical system. However, modeling of massively parallel numerical algorithms as switched dynamical systems results in a very large number of modes, which makes current analysis tools available for such systems computationally intractable. We develop new techniques that circumvent this scalability issue. The framework is presented on a one-dimensional heat equation as a case study and the proposed analysis framework is verified by solving the partial differential equation (PDE) in a $\mathtt{nVIDIA\: Tesla^{\scriptsize{TM}}}$ GPU machine, with asynchronous communication between cores.Comment: ACC 201

Simple CHT: A New Derivation of the Weakest Failure Detector for Consensus

The paper proposes an alternative proof that Omega, an oracle that outputs a process identifier and guarantees that eventually the same correct process identifier is output at all correct processes, provides minimal information about failures for solving consensus in read-write shared-memory systems: every oracle that gives enough failure information to solve consensus can be used to implement Omega. Unlike the original proof by Chandra, Hadzilacos and Toueg (CHT), the proof presented in this paper builds upon the very fact that 2-process wait-free consensus is impossible. Also, since the oracle that is used to implement can solve consensus, the implementation is allowed to directly access consensus objects. As a result, the proposed proof is shorter and conceptually simpler than the original one