Feedback stabilization of probabilistic finite state machines based on deep Q-network

BackgroundAs an important mathematical model, the finite state machine (FSM) has been used in many fields, such as manufacturing system, health care, and so on. This paper analyzes the current development status of FSMs. It is pointed out that the traditional methods are often inconvenient for analysis and design, or encounter high computational complexity problems when studying FSMs.MethodThe deep Q-network (DQN) technique, which is a model-free optimization method, is introduced to solve the stabilization problem of probabilistic finite state machines (PFSMs). In order to better understand the technique, some preliminaries, including Markov decision process, ϵ-greedy strategy, DQN, and so on, are recalled.ResultsFirst, a necessary and sufficient stabilizability condition for PFSMs is derived. Next, the feedback stabilization problem of PFSMs is transformed into an optimization problem. Finally, by using the stabilizability condition and deep Q-network, an algorithm for solving the optimization problem (equivalently, computing a state feedback stabilizer) is provided.DiscussionCompared with the traditional Q learning, DQN avoids the limited capacity problem. So our method can deal with high-dimensional complex systems efficiently. The effectiveness of our method is further demonstrated through an illustrative example

Minimalilty of Finite Automata Representation in Hybrid Systems Control

Abstract. This paper discusses a new approach to representing a finite automaton as a combination of a linear state equation with a smaller set of free binary variables (i.e., input variables) and binary inequalities, in order to reduce the computational time for solving the model predictive control problem of a class of hybrid systems. In particular, this paper is devoted to proving that a system representation derived by our proposed method is minimal in the sense that the number of its binary input variables is minimal among system models over all linear equivalence transformations that preserve the binary property of free (input) variables.