Finding Optimal Strategies in a Multi-Period Multi-Leader-Follower Stackelberg Game Using an Evolutionary Algorithm

Stackelberg games are a classic example of bilevel optimization problems, which are often encountered in game theory and economics. These are complex problems with a hierarchical structure, where one optimization task is nested within the other. Despite a number of studies on handling bilevel optimization problems, these problems still remain a challenging territory, and existing methodologies are able to handle only simple problems with few variables under assumptions of continuity and differentiability. In this paper, we consider a special case of a multi-period multi-leader-follower Stackelberg competition model with non-linear cost and demand functions and discrete production variables. The model has potential applications, for instance in aircraft manufacturing industry, which is an oligopoly where a few giant firms enjoy a tremendous commitment power over the other smaller players. We solve cases with different number of leaders and followers, and show how the entrance or exit of a player affects the profits of the other players. In the presence of various model complexities, we use a computationally intensive nested evolutionary strategy to find an optimal solution for the model. The strategy is evaluated on a test-suite of bilevel problems, and it has been shown that the method is successful in handling difficult bilevel problems.Comment: To be published in Computers and Operations Researc

Stackelberg Game for Distributed Time Scheduling in RF-Powered Backscatter Cognitive Radio Networks

In this paper, we study the transmission strategy adaptation problem in an RF-powered cognitive radio network, in which hybrid secondary users are able to switch between the harvest-then-transmit mode and the ambient backscatter mode for their communication with the secondary gateway. In the network, a monetary incentive is introduced for managing the interference caused by the secondary transmission with imperfect channel sensing. The sensing-pricing-transmitting process of the secondary gateway and the transmitters is modeled as a single-leader-multi-follower Stackelberg game. Furthermore, the follower sub-game among the secondary transmitters is modeled as a generalized Nash equilibrium problem with shared constraints. Based on our theoretical discoveries regarding the properties of equilibria in the follower sub-game and the Stackelberg game, we propose a distributed, iterative strategy searching scheme that guarantees the convergence to the Stackelberg equilibrium. The numerical simulations show that the proposed hybrid transmission scheme always outperforms the schemes with fixed transmission modes. Furthermore, the simulations reveal that the adopted hybrid scheme is able to achieve a higher throughput than the sum of the throughput obtained from the schemes with fixed transmission modes

Oracles & Followers: Stackelberg Equilibria in Deep Multi-Agent Reinforcement Learning

Stackelberg equilibria arise naturally in a range of popular learning problems, such as in security games or indirect mechanism design, and have received increasing attention in the reinforcement learning literature. We present a general framework for implementing Stackelberg equilibria search as a multi-agent RL problem, allowing a wide range of algorithmic design choices. We discuss how previous approaches can be seen as specific instantiations of this framework. As a key insight, we note that the design space allows for approaches not previously seen in the literature, for instance by leveraging multitask and meta-RL techniques for follower convergence. We propose one such approach using contextual policies, and evaluate it experimentally on both standard and novel benchmark domains, showing greatly improved sample efficiency compared to previous approaches. Finally, we explore the effect of adopting algorithm designs outside the borders of our framework

Prognostics-Based Two-Operator Competition for Maintenance and Service Part Logistics

Prognostics and timely maintenance of components are critical to the continuing operation of a system. By implementing prognostics, it is possible for the operator to maintain the system in the right place at the right time. However, the complexity in the real world makes near-zero downtime difficult to achieve partly because of a possible shortage of required service parts. This is realistic and quite important in maintenance practice. To coordinate with a prognostics-based maintenance schedule, the operator must decide when to order service parts and how to compete with other operators who also need the same parts. This research addresses a joint decision-making approach that assists two operators in making proactive maintenance decisions and strategically competing for a service part that both operators rely on for their individual operations. To this end, a maintenance policy involving competition in service part procurement is developed based on the Stackelberg game-theoretic model. Variations of the policy are formulated for three different scenarios and solved via either backward induction or genetic algorithm methods. Unlike the first two scenarios, the possibility for either of the operators being the leader in such competitions is considered in the third scenario. A numerical study on wind turbine operation is provided to demonstrate the use of the joint decision-making approach in maintenance and service part logistics