Optimal adaptive control for a class of stochastic systems

We study linear-quadratic adaptive tracking problems for a special class of stochastic systems expressed in the state-space form. This is a long-standing problem in the control of aircraft flying through atmospheric turbulence. Using an ELS-based algorithm and introducing dither in the control law we show that the resulting control achieves optimal cost in the limit, while simultaneously the unknown parameters converge to their true value

Optimal Universal Controllers for Roll Stabilization

Roll stabilization is an important problem of ship motion control. This problem becomes especially difficult if the same set of actuators (e.g. a single rudder) has to be used for roll stabilization and heading control of the vessel, so that the roll stabilizing system interferes with the ship autopilot. Finding the "trade-off" between the concurrent goals of accurate vessel steering and roll stabilization usually reduces to an optimization problem, which has to be solved in presence of an unknown wave disturbance. Standard approaches to this problem (loop-shaping, LQG, $H_{\infty}$-control etc.) require to know the spectral density of the disturbance, considered to be a \colored noise". In this paper, we propose a novel approach to optimal roll stabilization, approximating the disturbance by a polyharmonic signal with known frequencies yet uncertain amplitudes and phase shifts. Linear quadratic optimization problems in presence of polyharmonic disturbances can be solved by means of the theory of universal controllers developed by V.A. Yakubovich. An optimal universal controller delivers the optimal solution for any uncertain amplitudes and phases. Using Marine Systems Simulator (MSS) Toolbox that provides a realistic vessel's model, we compare our design method with classical approaches to optimal roll stabilization. Among three controllers providing the same quality of yaw steering, OUC stabilizes the roll motion most efficiently

Regret Minimization in Partially Observable Linear Quadratic Control

We study the problem of regret minimization in partially observable linear quadratic control systems when the model dynamics are unknown a priori. We propose ExpCommit, an explore-then-commit algorithm that learns the model Markov parameters and then follows the principle of optimism in the face of uncertainty to design a controller. We propose a novel way to decompose the regret and provide an end-to-end sublinear regret upper bound for partially observable linear quadratic control. Finally, we provide stability guarantees and establish a regret upper bound of O(T^(2/3)) for ExpCommit, where T is the time horizon of the problem

Sparse Wide-Area Control of Power Systems using Data-driven Reinforcement Learning

In this paper we present an online wide-area oscillation damping control (WAC) design for uncertain models of power systems using ideas from reinforcement learning. We assume that the exact small-signal model of the power system at the onset of a contingency is not known to the operator and use the nominal model and online measurements of the generator states and control inputs to rapidly converge to a state-feedback controller that minimizes a given quadratic energy cost. However, unlike conventional linear quadratic regulators (LQR), we intend our controller to be sparse, so its implementation reduces the communication costs. We, therefore, employ the gradient support pursuit (GraSP) optimization algorithm to impose sparsity constraints on the control gain matrix during learning. The sparse controller is thereafter implemented using distributed communication. Using the IEEE 39-bus power system model with 1149 unknown parameters, it is demonstrated that the proposed learning method provides reliable LQR performance while the controller matched to the nominal model becomes unstable for severely uncertain systems.Comment: Submitted to IEEE ACC 2019. 8 pages, 4 figure