Asynchronous Parallel Stochastic Gradient Descent - A Numeric Core for Scalable Distributed Machine Learning Algorithms

The implementation of a vast majority of machine learning (ML) algorithms boils down to solving a numerical optimization problem. In this context, Stochastic Gradient Descent (SGD) methods have long proven to provide good results, both in terms of convergence and accuracy. Recently, several parallelization approaches have been proposed in order to scale SGD to solve very large ML problems. At their core, most of these approaches are following a map-reduce scheme. This paper presents a novel parallel updating algorithm for SGD, which utilizes the asynchronous single-sided communication paradigm. Compared to existing methods, Asynchronous Parallel Stochastic Gradient Descent (ASGD) provides faster (or at least equal) convergence, close to linear scaling and stable accuracy

Optimal Decentralized Protocols for Electric Vehicle Charging

We propose decentralized algorithms for optimally scheduling electric vehicle charging. The algorithms exploit the elasticity and controllability of electric vehicle related loads in order to fill the valleys in electric demand profile. We formulate a global optimization problem whose objective is to impose a generalized notion of valley-filling, study properties of the optimal charging profiles, and give decentralized offline and online algorithms to solve the problem. In each iteration of the proposed algorithms, electric vehicles choose their own charging profiles for the rest horizon according to the price profile broadcast by the utility, and the utility updates the price profile to guide their behavior. The offline algorithms are guaranteed to converge to optimal charging profiles irrespective of the specifications (e.g., maximum charging rate and deadline) of electric vehicles at the expense of a restrictive assumption that all electric vehicles are available for negotiation at the beginning of the planning horizon. The online algorithms relax this assumption by using a scalar prediction of future total charging demand at each time instance and yield near optimal charging profiles. The proposed algorithms need no coordination among the electric vehicles, hence their implementation requires low communication and computation capability. Simulation results are provided to support these results