Softpressure: A Schedule-Driven Backpressure Algorithm for Coping with Network Congestion

We consider the problem of minimizing the delay of jobs moving through a directed graph of service nodes. In this problem, each node may have several links and is constrained to serve one link at a time. As jobs move through the network, they can pass through a node only after they have been serviced by that node. The objective is to minimize the delay jobs incur sitting in queues waiting to be serviced. Two distinct approaches to this problem have emerged from respective work in queuing theory and dynamic scheduling: the backpressure algorithm and schedule-driven control. In this paper, we present a hybrid approach of those two methods that incorporates the stability of queuing theory into a schedule-driven control framework. We then demonstrate how this hybrid method outperforms the other two in a real-time traffic signal control problem, where the nodes are traffic lights, the links are roads, and the jobs are vehicles. We show through simulations that, in scenarios with heavy congestion, the hybrid method results in 50% and 15% reductions in delay over schedule-driven control and backpressure respectively. A theoretical analysis also justifies our results.Comment: IJCAI 201

Work Capacity of Freelance Markets: Fundamental Limits and Decentralized Schemes

Crowdsourcing of jobs to online freelance markets is rapidly gaining popularity. Most crowdsourcing platforms are uncontrolled and offer freedom to customers and freelancers to choose each other. This works well for unskilled jobs (e.g., image classification) with no specific quality requirement since freelancers are functionally identical. For skilled jobs (e.g., software development) with specific quality requirements, however, this does not ensure that the maximum number of job requests is satisfied. In this work we determine the capacity of freelance markets, in terms of maximum satisfied job requests, and propose centralized schemes that achieve capacity. To ensure decentralized operation and freedom of choice for customers and freelancers, we propose simple schemes compatible with the operation of current crowdsourcing platforms that approximately achieve capacity. Further, for settings where the number of job requests exceeds capacity, we propose a scheme that is agnostic of that information, but is optimal and fair in declining jobs without wait

Decentralized Q-Learning for Stochastic Teams and Games

There are only a few learning algorithms applicable to stochastic dynamic teams and games which generalize Markov decision processes to decentralized stochastic control problems involving possibly self-interested decision makers. Learning in games is generally difficult because of the non-stationary environment in which each decision maker aims to learn its optimal decisions with minimal information in the presence of the other decision makers who are also learning. In stochastic dynamic games, learning is more challenging because, while learning, the decision makers alter the state of the system and hence the future cost. In this paper, we present decentralized Q-learning algorithms for stochastic games, and study their convergence for the weakly acyclic case which includes team problems as an important special case. The algorithm is decentralized in that each decision maker has access to only its local information, the state information, and the local cost realizations; furthermore, it is completely oblivious to the presence of other decision makers. We show that these algorithms converge to equilibrium policies almost surely in large classes of stochastic games.Comment: To appear in IEEE Trans. Automatic Contro

Enhanced Mobility With Connectivity and Automation: A Review of Shared Autonomous Vehicle Systems

Shared mobility can provide access to transportation on a custom basis without vehicle ownership. The advent of connected and automated vehicle technologies can further enhance the potential benefits of shared mobility systems. Although the implications of a system with shared autonomous vehicles have been investigated, the research reported in the literature has exhibited contradictory outcomes. In this paper, we present a summary of the research efforts in shared autonomous vehicle systems that have been reported in the literature to date and discuss potential future research directions.Comment: 17 pages, 3 figures, IEEE Intelligent Transportation Systems Magazine, 202

Decentralized Learning for Optimality in Stochastic Dynamic Teams and Games with Local Control and Global State Information

Stochastic dynamic teams and games are rich models for decentralized systems and challenging testing grounds for multi-agent learning. Previous work that guaranteed team optimality assumed stateless dynamics, or an explicit coordination mechanism, or joint-control sharing. In this paper, we present an algorithm with guarantees of convergence to team optimal policies in teams and common interest games. The algorithm is a two-timescale method that uses a variant of Q-learning on the finer timescale to perform policy evaluation while exploring the policy space on the coarser timescale. Agents following this algorithm are "independent learners": they use only local controls, local cost realizations, and global state information, without access to controls of other agents. The results presented here are the first, to our knowledge, to give formal guarantees of convergence to team optimality using independent learners in stochastic dynamic teams and common interest games

Learning in Multi-level Stochastic games with Delayed Information

Distributed decision-makers are modeled as players in a game with two levels. High level decisions concern the game environment and determine the willingness of the players to form a coalition (or group). Low level decisions involve the actions to be implemented within the chosen environment. Coalition and action strategies are determined by probability distributions, which are updated using learning automata schemes. The payoffs are also probabilistic and there is uncertainty in the state vector since information is delayed. The goal is to reach equilibrium in both levels of decision making; the results show the conditions for instability, based on the age of information.Comment: Appears in Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence (UAI1994

Optimal Routing for Delay-Sensitive Traffic in Overlay Networks

We design dynamic routing policies for an overlay network which meet delay requirements of real-time traffic being served on top of an underlying legacy network, where the overlay nodes do not know the underlay characteristics. We pose the problem as a constrained MDP, and show that when the underlay implements static policies such as FIFO with randomized routing, then a decentralized policy, that can be computed efficiently in a distributed fashion, is optimal. Our algorithm utilizes multi-timescale stochastic approximation techniques, and its convergence relies on the fact that the recursions asymptotically track a nonlinear differential equation, namely the replicator equation. Extensive simulations show that the proposed policy indeed outperforms the existing policies

A Survey and Taxonomy of Urban Traffic Management: Towards Vehicular Networks

Urban Traffic Management (UTM) topics have been tackled since long time, mainly by civil engineers and by city planners. The introduction of new communication technologies - such as cellular systems, satellite positioning systems and inter-vehicle communications - has significantly changed the way researchers deal with UTM issues. In this survey, we provide a review and a classification of how UTM has been addressed in the literature. We start from the recent achievements of "classical" approaches to urban traffic estimation and optimization, including methods based on the analysis of data collected by fixed sensors (e.g., cameras and radars), as well as methods based on information provided by mobile phones, such as Floating Car Data (FCD). Afterwards, we discuss urban traffic optimization, presenting the most recent works on traffic signal control and vehicle routing control. Then, after recalling the main concepts of Vehicular Ad-Hoc Networks (VANETs), we classify the different VANET-based approaches to UTM, according to three categories ("pure" VANETs, hybrid vehicular-sensor networks and hybrid vehicular-cellular networks), while illustrating the major research issues for each of them. The main objective of this survey is to provide a comprehensive view on UTM to researchers with focus on VANETs, in order to pave the way for the design and development of novel techniques for mitigating urban traffic problems, based on inter-vehicle communications

Efficient and Flexible Crowdsourcing of Specialized Tasks with Precedence Constraints

Many companies now use crowdsourcing to leverage external (as well as internal) crowds to perform specialized work, and so methods of improving efficiency are critical. Tasks in crowdsourcing systems with specialized work have multiple steps and each step requires multiple skills. Steps may have different flexibilities in terms of obtaining service from one or multiple agents, due to varying levels of dependency among parts of steps. Steps of a task may have precedence constraints among them. Moreover, there are variations in loads of different types of tasks requiring different skill-sets and availabilities of different types of agents with different skill-sets. Considering these constraints together necessitates the design of novel schemes to allocate steps to agents. In addition, large crowdsourcing systems require allocation schemes that are simple, fast, decentralized and offer customers (task requesters) the freedom to choose agents. In this work we study the performance limits of such crowdsourcing systems and propose efficient allocation schemes that provably meet the performance limits under these additional requirements. We demonstrate our algorithms on data from a crowdsourcing platform run by a non-profit company and show significant improvements over current practice

A BCMP Network Approach to Modeling and Controlling Autonomous Mobility-on-Demand Systems

In this paper we present a queueing network approach to the problem of routing and rebalancing a fleet of self-driving vehicles providing on-demand mobility within a capacitated road network. We refer to such systems as autonomous mobility-on-demand systems, or AMoD. We first cast an AMoD system into a closed, multi-class BCMP queueing network model. Second, we present analysis tools that allow the characterization of performance metrics for a given routing policy, in terms, e.g., of vehicle availabilities, and first and second order moments of vehicle throughput. Third, we propose a scalable method for the synthesis of routing policies, with performance guarantees in the limit of large fleet sizes. Finally, we validate our theoretical results on a case study of New York City. Collectively, this paper provides a unifying framework for the analysis and control of AMoD systems, which subsumes earlier Jackson and network flow models, provides a quite large set of modeling options (e.g., the inclusion of road capacities and general travel time distributions), and allows the analysis of second and higher-order moments for the performance metrics.Comment: 18 pages, 3 figures. In preparation for conference submission. In version 2, clarity is improved and some typos are removed with no changes to the technical content of the pape