Avoiding Braess' Paradox through Collective Intelligence

In an Ideal Shortest Path Algorithm (ISPA), at each moment each router in a network sends all of its traffic down the path that will incur the lowest cost to that traffic. In the limit of an infinitesimally small amount of traffic for a particular router, its routing that traffic via an ISPA is optimal, as far as cost incurred by that traffic is concerned. We demonstrate though that in many cases, due to the side-effects of one router's actions on another routers performance, having routers use ISPA's is suboptimal as far as global aggregate cost is concerned, even when only used to route infinitesimally small amounts of traffic. As a particular example of this we present an instance of Braess' paradox for ISPA's, in which adding new links to a network decreases overall throughput. We also demonstrate that load-balancing, in which the routing decisions are made to optimize the global cost incurred by all traffic currently being routed, is suboptimal as far as global cost averaged across time is concerned. This is also due to "side-effects", in this case of current routing decision on future traffic. The theory of COllective INtelligence (COIN) is concerned precisely with the issue of avoiding such deleterious side-effects. We present key concepts from that theory and use them to derive an idealized algorithm whose performance is better than that of the ISPA, even in the infinitesimal limit. We present experiments verifying this, and also showing that a machine-learning-based version of this COIN algorithm in which costs are only imprecisely estimated (a version potentially applicable in the real world) also outperforms the ISPA, despite having access to less information than does the ISPA. In particular, this COIN algorithm avoids Braess' paradox.Comment: 28 page

Payoff-Based Dynamics for Multiplayer Weakly Acyclic Games

We consider repeated multiplayer games in which players repeatedly and simultaneously choose strategies from a finite set of available strategies according to some strategy adjustment process. We focus on the specific class of weakly acyclic games, which is particularly relevant for multiagent cooperative control problems. A strategy adjustment process determines how players select their strategies at any stage as a function of the information gathered over previous stages. Of particular interest are “payoff-based” processes in which, at any stage, players know only their own actions and (noise corrupted) payoffs from previous stages. In particular, players do not know the actions taken by other players and do not know the structural form of payoff functions. We introduce three different payoff-based processes for increasingly general scenarios and prove that, after a sufficiently large number of stages, player actions constitute a Nash equilibrium at any stage with arbitrarily high probability. We also show how to modify player utility functions through tolls and incentives in so-called congestion games, a special class of weakly acyclic games, to guarantee that a centralized objective can be realized as a Nash equilibrium. We illustrate the methods with a simulation of distributed routing over a network

Routing choices in intelligent transport systems

Road congestion is a phenomenon that can often be avoided; roads become popular, travel times increase, which could be mitigated with better coordination mechanisms. The choice of route, mode of transport, and departure time all play a crucial part in controlling congestion levels. Technology, such as navigation applications, have the ability to influence these decisions and play an essential role in congestion reduction. To predict vehicles' routing behaviours, we model the system as a game with rational players. Players choose a path between origin and destination nodes in a network. Each player seeks to minimise their own journey time, often leading to inefficient equilibria with poor social welfare. Traffic congestion motivates the results in this thesis. However, the results also hold true for many other applications where congestion occurs, e.g. power grid demand. Coordinating route selection to reduce congestion constitutes a social dilemma for vehicles. In sequential social dilemmas, players' strategies need to balance their vulnerability to exploitation from their opponents and to learn to cooperate to achieve maximal payouts. We address this trade-off between mathematical safety and cooperation of strategies in social dilemmas to motivate our proposed algorithm, a safe method of achieving cooperation in social dilemmas, including route choice games. Many vehicles use navigation applications to help plan their journeys, but these provide only partial information about the routes available to them. We find a class of networks for which route information distribution cannot harm the receiver's expected travel times. Additionally, we consider a game where players always follow the route chosen by an application or where vehicle route selection is controlled by a route planner, such as autonomous vehicles. We show that having multiple route planners controlling vehicle routing leads to inefficient equilibria. We calculate the Price of Anarchy (PoA) for polynomial function travel times and show that multiagent reinforcement learning algorithms suffer from the predicted Price of Anarchy when controlling vehicle routing. Finally, we equip congestion games with waiting times at junctions to model the properties of traffic lights at intersections. Here, we show that Braess' paradox can be avoided by implementing traffic light cycles and establish the PoA for realistic waiting times. By employing intelligent traffic lights that use myopic learning, such as multi-agent reinforcement learning, we prove a natural reward function guarantees convergence to equilibrium. Moreover, we highlight the impact of multi-agent reinforcement learning traffic lights on the fairness of journey times to vehicles