4 research outputs found
A Switched Dynamical System Framework for Analysis of Massively Parallel Asynchronous Numerical Algorithms
In the near future, massively parallel computing systems will be necessary to
solve computation intensive applications. The key bottleneck in massively
parallel implementation of numerical algorithms is the synchronization of data
across processing elements (PEs) after each iteration, which results in
significant idle time. Thus, there is a trend towards relaxing the
synchronization and adopting an asynchronous model of computation to reduce
idle time. However, it is not clear what is the effect of this relaxation on
the stability and accuracy of the numerical algorithm. In this paper we present
a new framework to analyze such algorithms. We treat the computation in each PE
as a dynamical system and model the asynchrony as stochastic switching. The
overall system is then analyzed as a switched dynamical system. However,
modeling of massively parallel numerical algorithms as switched dynamical
systems results in a very large number of modes, which makes current analysis
tools available for such systems computationally intractable. We develop new
techniques that circumvent this scalability issue. The framework is presented
on a one-dimensional heat equation as a case study and the proposed analysis
framework is verified by solving the partial differential equation (PDE) in a
GPU machine, with asynchronous
communication between cores.Comment: ACC 201
Convergence Analysis of Asynchronous Consensus in Discrete-time Multi-agent Systems with Fixed Topology
In this paper, we study a convergence condition for asynchronous consensus
problems in multi-agent systems. The convergence in this context implies the
asynchronous consensus value converges to the synchronous one and is unique.
Although it is reported in the literature that the consensus value under
asynchronous communications may not coincide with the synchronous consensus
value, it has not received much attention. In some applications, the
discrepancy between them may result in serious consequences. For such
applications it is critical to determine under what conditions the asynchronous
consensus value is the same as the synchronous consensus value. We illustrate
these issues with a few examples and then provide a condition, which guarantees
that the asynchronous consensus value converges to the synchronous one. The
validity of the proposed result is verified with simulations
On the Convergence Analysis of Asynchronous Distributed Quadratic Programming via Dual Decomposition
In this paper, we analyze the convergence as well as the rate of convergence
of asynchronous distributed quadratic programming (QP) with dual decomposition
technique. In general, distributed optimization requires synchronization of
data at each iteration step due to the interdependency of data. This
synchronization latency may incur a large amount of waiting time caused by an
idle process during computation. We aim to attack this synchronization penalty
in distributed QP problems by implementing asynchronous update of dual
variable. The price to pay for adopting asynchronous computing algorithms is
unpredictability of the solution, resulting in a tradeoff between speedup and
accuracy. Thus, the convergence to an optimal solution is not guaranteed owing
to the stochastic behavior of asynchrony. In this paper, we employ the switched
system framework as an analysis tool to investigate the convergence of
asynchronous distributed QP. This switched system will facilitate analysis on
asynchronous distributed QP with dual decomposition, providing necessary and
sufficient conditions for the mean square convergence. Also, we provide an
analytic expression for the rate of convergence through the switched system,
which enables performance analysis of asynchronous algorithms as compared with
synchronous case. To verify the validity of the proposed methods, numerical
examples are presented with an implementation of asynchronous parallel QP using
OpenMP
A Stabilizing Control Algorithm for Asynchronous Parallel Quadratic Programming via Dual Decomposition
This paper proposes a control algorithm for stable implementation of
asynchronous parallel quadratic programming (PQP) through dual decomposition
technique. In general, distributed and parallel optimization requires
synchronization of data at each iteration step due to the interdependency of
data. The synchronization latency may incur a large amount of waiting time
caused by an idle process during computation. We aim to mitigate this
synchronization penalty in PQP problems by implementing asynchronous updates of
the dual variable. The price to pay for adopting asynchronous computing
algorithms is the unpredictability of the solution, resulting in a tradeoff
between speedup and accuracy. In the worst case, the state of interest may
become unstable owing to the stochastic behavior of asynchrony. We investigate
the stability condition of asynchronous PQP problems by employing the switched
system framework. A formal algorithm is provided to ensure the asymptotic
stability of dual variables. Further, it is shown that the implementation of
the proposed algorithm guarantees the uniqueness of optimal solutions,
irrespective of asynchronous behavior. To verify the validity of the proposed
methods, simulation results are presented.Comment: arXiv admin note: substantial text overlap with arXiv:1506.0548