Game Theoretic Approach to the Stabilization of Heterogeneous Multiagent Systems Using Subsidy

We consider a multiagent system consisting of selfish and heterogeneous agents. Its behavior is modeled by multipopulation replicator dynamics, where payoff functions of populations are different from each other. In general, there exist several equilibrium points in the replicator dynamics. In order to stabilize a desirable equilibrium point, we introduce a controller called a government which controls the behaviors of agents by offering them subsidies. In previous work, it is assumed that the government determines the subsidies based on the populations the agents belong to. In general, however, the government cannot identify the members of each population. In this paper, we assume that the government observes the action of each agent and determines the subsidies based on the observed action profile. Then, we model the controlled behaviors of the agents using replicator dynamics with feedback. We derive a stabilization condition of the target equilibrium point in the replicator dynamics.Comment: 6 pages, IEEE Conference on Decision and Control, 201

Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxation

Deep reinforcement learning (DRL) has attracted much attention as an approach to solve optimal control problems without mathematical models of systems. On the other hand, in general, constraints may be imposed on optimal control problems. In this study, we consider the optimal control problems with constraints to complete temporal control tasks. We describe the constraints using signal temporal logic (STL), which is useful for time sensitive control tasks since it can specify continuous signals within bounded time intervals. To deal with the STL constraints, we introduce an extended constrained Markov decision process (CMDP), which is called a $\tau$-CMDP. We formulate the STL-constrained optimal control problem as the $\tau$-CMDP and propose a two-phase constrained DRL algorithm using the Lagrangian relaxation method. Through simulations, we also demonstrate the learning performance of the proposed algorithm.Comment: 16 pages, 20 figures, accepted for IEEE Acces

Hyper-Labeled Transition System and Its Application to Planning Under Linear Temporal Logic Constraints

Recently, formal methods have been paid much attention to in planning. We often leverage a labeled transition system (LTS) with a set of atomic propositions as a model of an environment. However, for example, in a flexible manufacturing system where an assembling machine has several functions that are performed exclusively, we need several states labeled by different sets of atomic propositions to discriminate the functions explicitly. This letter aims to reduce the number of the states. We introduce an extension of the LTS, called a hyperLTS, the labeling function of which assigns a set of sets of atomic propositions to each state. Then, we propose linear encodings for the hyperLTS to represent a sequence of pairs of a state and a selected set of atomic propositions. The hyperLTS-based modeling is consequently applied to a planning problem with one hard constraint and several soft constraints, thereby converting it into an integer linear programming problem. The effectiveness of the proposed modeling is illustrated through an example of a path planning problem of a mobile robot in a manufacturing system.Takuma Kinugawa and Toshimitsu Ushio. Hyper-Labeled Transition System and Its Application to Planning Under Linear Temporal Logic Constraints. IEEE Control Systems Letters, 6, pp. 2437-2442, 2022. https://doi.org/10.1109/LCSYS.2022.3162313