5,965 research outputs found
An efficient and robust decentralized algorithm for detecting the global convergence in asynchronous iterative algorithms
URL : http://vecpar.fe.up.pt/2008/papers/25.pdfInternational audienceIn this paper we present a practical, efficient and robust algorithm for detecting the global convergence in any asynchronous iterative process. A proven theoretical version, together with a first practical version, was presented in [1]. However, the main drawback of that first practical version was to require the determination of the maximal communication time between any couple of nodes in the system during the entire iterative process. The version presented in this paper does not require any additional information on the parallel system while always ensuring correct detections
Real and Complex Monotone Communication Games
Noncooperative game-theoretic tools have been increasingly used to study many
important resource allocation problems in communications, networking, smart
grids, and portfolio optimization. In this paper, we consider a general class
of convex Nash Equilibrium Problems (NEPs), where each player aims to solve an
arbitrary smooth convex optimization problem. Differently from most of current
works, we do not assume any specific structure for the players' problems, and
we allow the optimization variables of the players to be matrices in the
complex domain. Our main contribution is the design of a novel class of
distributed (asynchronous) best-response- algorithms suitable for solving the
proposed NEPs, even in the presence of multiple solutions. The new methods,
whose convergence analysis is based on Variational Inequality (VI) techniques,
can select, among all the equilibria of a game, those that optimize a given
performance criterion, at the cost of limited signaling among the players. This
is a major departure from existing best-response algorithms, whose convergence
conditions imply the uniqueness of the NE. Some of our results hinge on the use
of VI problems directly in the complex domain; the study of these new kind of
VIs also represents a noteworthy innovative contribution. We then apply the
developed methods to solve some new generalizations of SISO and MIMO games in
cognitive radios and femtocell systems, showing a considerable performance
improvement over classical pure noncooperative schemes.Comment: to appear on IEEE Transactions in Information Theor
Asynchronous Optimization Methods for Efficient Training of Deep Neural Networks with Guarantees
Asynchronous distributed algorithms are a popular way to reduce
synchronization costs in large-scale optimization, and in particular for neural
network training. However, for nonsmooth and nonconvex objectives, few
convergence guarantees exist beyond cases where closed-form proximal operator
solutions are available. As most popular contemporary deep neural networks lead
to nonsmooth and nonconvex objectives, there is now a pressing need for such
convergence guarantees. In this paper, we analyze for the first time the
convergence of stochastic asynchronous optimization for this general class of
objectives. In particular, we focus on stochastic subgradient methods allowing
for block variable partitioning, where the shared-memory-based model is
asynchronously updated by concurrent processes. To this end, we first introduce
a probabilistic model which captures key features of real asynchronous
scheduling between concurrent processes; under this model, we establish
convergence with probability one to an invariant set for stochastic subgradient
methods with momentum.
From the practical perspective, one issue with the family of methods we
consider is that it is not efficiently supported by machine learning
frameworks, as they mostly focus on distributed data-parallel strategies. To
address this, we propose a new implementation strategy for shared-memory based
training of deep neural networks, whereby concurrent parameter servers are
utilized to train a partitioned but shared model in single- and multi-GPU
settings. Based on this implementation, we achieve on average 1.2x speed-up in
comparison to state-of-the-art training methods for popular image
classification tasks without compromising accuracy
Relational reasoning via probabilistic coupling
Probabilistic coupling is a powerful tool for analyzing pairs of
probabilistic processes. Roughly, coupling two processes requires finding an
appropriate witness process that models both processes in the same probability
space. Couplings are powerful tools proving properties about the relation
between two processes, include reasoning about convergence of distributions and
stochastic dominance---a probabilistic version of a monotonicity property.
While the mathematical definition of coupling looks rather complex and
cumbersome to manipulate, we show that the relational program logic pRHL---the
logic underlying the EasyCrypt cryptographic proof assistant---already
internalizes a generalization of probabilistic coupling. With this insight,
constructing couplings is no harder than constructing logical proofs. We
demonstrate how to express and verify classic examples of couplings in pRHL,
and we mechanically verify several couplings in EasyCrypt
Certified Reinforcement Learning with Logic Guidance
This paper proposes the first model-free Reinforcement Learning (RL)
framework to synthesise policies for unknown, and continuous-state Markov
Decision Processes (MDPs), such that a given linear temporal property is
satisfied. We convert the given property into a Limit Deterministic Buchi
Automaton (LDBA), namely a finite-state machine expressing the property.
Exploiting the structure of the LDBA, we shape a synchronous reward function
on-the-fly, so that an RL algorithm can synthesise a policy resulting in traces
that probabilistically satisfy the linear temporal property. This probability
(certificate) is also calculated in parallel with policy learning when the
state space of the MDP is finite: as such, the RL algorithm produces a policy
that is certified with respect to the property. Under the assumption of finite
state space, theoretical guarantees are provided on the convergence of the RL
algorithm to an optimal policy, maximising the above probability. We also show
that our method produces ''best available'' control policies when the logical
property cannot be satisfied. In the general case of a continuous state space,
we propose a neural network architecture for RL and we empirically show that
the algorithm finds satisfying policies, if there exist such policies. The
performance of the proposed framework is evaluated via a set of numerical
examples and benchmarks, where we observe an improvement of one order of
magnitude in the number of iterations required for the policy synthesis,
compared to existing approaches whenever available.Comment: This article draws from arXiv:1801.08099, arXiv:1809.0782
- …