Exploiting Symmetry in Dynamics for Model-Based Reinforcement Learning With Asymmetric Rewards

Recent work in reinforcement learning has leveraged symmetries in the model to improve sample efficiency in training a policy. A commonly used simplifying assumption is that the dynamics and reward both exhibit the same symmetry; however, in many real-world environments, the dynamical model exhibits symmetry independent of the reward model. In this letter, we assume only the dynamics exhibit symmetry, extending the scope of problems in reinforcement learning and learning in control theory to which symmetry techniques can be applied. We use Cartan's moving frame method to introduce a technique for learning dynamics that, by construction, exhibit specified symmetries. Numerical experiments demonstrate that the proposed method learns a more accurate dynamical model

Certifying Stability and Performance of Uncertain Differential-Algebraic Systems: A Dissipativity Framework

This paper presents a novel framework for characterizing dissipativity of uncertain dynamical systems subject to algebraic constraints. The main results provide sufficient conditions for dissipativity when uncertainties are characterized by integral quadratic constraints. For polynomial or linear dynamics, these conditions can be efficiently verified through sum-of-squares or semidefinite programming. The practical impact of this work is illustrated through a case study that examines performance of the IEEE 39-bus power network with uncertainties used to model a set of potential line failures

Synthesis of Stabilizing Recurrent Equilibrium Network Controllers

We propose a parameterization of a nonlinear dynamic controller based on the recurrent equilibrium network, a generalization of the recurrent neural network. We derive constraints on the parameterization under which the controller guarantees exponential stability of a partially observed dynamical system with sector-bounded nonlinearities. Finally, we present a method to synthesize this controller using projected policy gradient methods to maximize a reward function with arbitrary structure. The projection step involves the solution of convex optimization problems. We demonstrate the proposed method with simulated examples of controlling nonlinear plants, including plants modeled with neural networks.Comment: Submitted to IEEE CDC 2022. arXiv admin note: text overlap with arXiv:2109.0386

Grouping of N−1N-1 Contingencies for Controller Synthesis: A Study for Power Line Failures

The problem of maintaining power system stability and performance after the failure of any single line in a power system (an “$N-1$ contingency”) is investigated. Due to the large number of possible $N-1$ contingencies for a power network, it is impractical to optimize controller parameters for each possible contingency a priori. A method to partition a set of contingencies into groups of contingencies that are similar to each other from a control perspective is presented. Design of a single controller for each group, rather than for each contingency, provides a computationally tractable method for maintaining stability and performance after element failures. The choice of number of groups tunes a trade-off between computation time and controller performance for a given set of contingencies. Results are simulated on the IEEE 39-bus and 68-bus systems, illustrating that, with controllers designed for a relatively small number of groups, power system stability may be significantly improved after an $N-1$ contingency compared to continued use of the nominal controller. Furthermore, performance is comparable to that of controllers designed for each contingency individually