The usual robust control framework in discrete time: some interesting results

By applying robust control the decision maker wants to make good decisions when his model is only a good approximation of the true one. Such decisions are said to be robust to model misspecification. In this paper it is shown that the application of the usual robust control framework in discrete time problems is associated with some interesting, if not unexpected, results. Results that have far reaching consequences when robust control is applied sequentially, say every year in fiscal policy or every quarter (month) in monetary policy. This is true when unstructured uncertainty à la Hansen and Sargent is used, both in the case of a “probabilistically sophisticated” and a non-“probabilistically sophisticated” decision maker, or when uncertainty is related to unknown structural parameters of the model

How Robust is Robust Control in the Time Domain?

By applying robust control the decision maker wants to make good decisions when his model is only a good approximation of the true one. Such decisions are said to be robust to model misspecification. In this paper it is shown that both a “probabilistically sophisticated” and a non-“probabilistically sophisticated” decision maker applying robust control in the time domain are indeed assuming a very special kind of “misspecification of the approximating model.” This is true when unstructured uncertainty à la Hansen and Sargent is used or when uncertainty is related to unknown structural parameters of the modelLinear quadratic tracking problem, optimal control, robust control, time-varying parameters

On the Convergence of No-Regret Learning Dynamics in Time-Varying Games

Most of the literature on learning in games has focused on the restrictive setting where the underlying repeated game does not change over time. Much less is known about the convergence of no-regret learning algorithms in dynamic multiagent settings. In this paper, we characterize the convergence of optimistic gradient descent (OGD) in time-varying games. Our framework yields sharp convergence bounds for the equilibrium gap of OGD in zero-sum games parameterized on natural variation measures of the sequence of games, subsuming known results for static games. Furthermore, we establish improved second-order variation bounds under strong convexity-concavity, as long as each game is repeated multiple times. Our results also apply to time-varying general-sum multi-player games via a bilinear formulation of correlated equilibria, which has novel implications for meta-learning and for obtaining refined variation-dependent regret bounds, addressing questions left open in prior papers. Finally, we leverage our framework to also provide new insights on dynamic regret guarantees in static games.Comment: To appear at NeurIPS 2023; V3 incorporates reviewers' feedback and minor correction

Game Theoretic Strategies for Spacecraft Rendezvous and Motion Synchronization

Uno dei possibili sviluppi della guida e del controllo relativo nello spazio è quella di estendere gli algoritmi per operazioni di rendezvous e di docking autonome a più veicoli spaziali che collaborano tra di loro. Il problema del rendezvous tra due veicoli spaziali viene risolto utilizzando la teoria dei giochi differenziali lineari quadratici. La dinamica del gioco viene descritta in un sistema di riferimento cartesiano non inerziale. Per estendere l'utilizzo della teoria dei giochi differenziali lineari quadratici alle equazioni non lineari di moto relativo è stata utilizzata le tecnica di parametrizzazione in funzione dello stato o linearizzazione estesa. Nelle simulazioni è stato valutato il confronto tra le prestazioni e le traiettorie ottenute con l'equilibrio di Pareto e quello di Nash quando entrambi i veicoli spaziali agiscono sotto spinta continua. Una strategia sequenziale è stata utilizzata per estendere il gioco differenziali a più di due giocatori per avere la sincronizzazione del moto relativo durante operazioni di assemblaggio nello spazio. One of the main challenges for autonomous spacecraft relative guidance and control is extending the algorithms for autonomous rendezvous and docking (AR&D) operations to multiple collaborative spacecraft. In this thesis, the autonomous rendezvous problem, between two active spacecraft, is formulated as a two player nonzero-sum differential game. The local-vertical local-horizontal (LVLH) rotating reference frame is used to describe the dynamic of the game. The State-Dependent Riccati equation (SDRE) method is applied to extend the Linear Quadratic differential game theory to obtain a feedback control law for nonlinear equation of relative motion. In the simulations both the spacecraft use continuous thrust engines. A comparison among Pareto and Nash equilibrium has been performed. A multiplayer sequential game strategy is used to extend the control law to many spacecraft for relative motion synchronization in an on-orbit self assembly strategy

Realtime Motion Planning for Manipulator Robots under Dynamic Environments: An Optimal Control Approach

This report presents optimal control methods integrated with hierarchical control framework to realize real-time collision-free optimal trajectories for motion control in kinematic chain manipulator (KCM) robot systems under dynamic environments. Recently, they have been increasingly used in applications where manipulators are required to interact with random objects and humans. As a result, more complex trajectory planning schemes are required. The main objective of this research is to develop new motion control strategies that can enable such robots to operate efficiently and optimally in such unknown and dynamic environments. Two direct optimal control methods: The direct collocation method and discrete mechanics for optimal control methods are investigated for solving the related constrained optimal control problem and the results are compared. Using the receding horizon control structure, open-loop sub-optimal trajectories are generated as real-time input to the controller as opposed to the predefined trajectory over the entire time duration. This, in essence, captures the dynamic nature of the obstacles. The closed-loop position controller is then engaged to span the robot end-effector along this desired optimal path by computing appropriate torque commands for the joint actuators. Employing a two-degree of freedom technique, collision-free trajectories and robot environment information are transmitted in real-time by the aid of a bidirectional connectionless datagram transfer. A hierarchical network control platform is designed to condition triggering of precedent activities between a dedicated machine computing the optimal trajectory and the real-time computer running a low-level controller. Experimental results on a 2-link planar robot are presented to validate the main ideas. Real-time implementation of collision-free workspace trajectory control is achieved for cases where obstacles are arbitrarily changing in the robot workspace

Koopman Operator Theory and The Applied Perspective of Modern Data-Driven Systems

Recent theoretical developments in dynamical systems and machine learning have allowed researchers to re-evaluate how dynamical systems are modeled and controlled. In this thesis, Koopman operator theory is used to model dynamical systems and obtain optimal control solutions for nonlinear systems using sampled system data. The Koopman operator is obtained using data generated from a real physical system or from an analytical model which describes the physical system under nominal conditions. One of the critical advantages of the Koopman operator is that the response of the nonlinear system can be obtained from an equivalent infinite dimensional linear system. This is achieved by exploiting the topological structure associated with the spectrum of the Koopman operator and the Koopman eigenfunctions. The main contributions of this thesis are threefold. First, we provide a data-driven approach for system identification, and a model-based approach for obtaining an analytic change of coordinates associated with the principle Koopman eigenfunctions for systems with hyperbolic equilibrium points. A new derivation of the Hamilton-Jacobi equations associated with the infinite time horizon nonlinear optimal control problem is obtained using the Koopman generator. Then, a learning algorithm called Koopman Policy Iteration is used to obtain the solution to the infinite horizon nonlinear optimal fixed point regulation problem without state and input constraints. Finally, the finite time nonlinear optimal control problem with state and input constraints is solved using a receding horizon optimization approach called dual mode model predictive control using Koopman eigenfunctions. Evidence supporting the convergence of these methods are provided using analytical examples