Symplectic partitioned Runge-Kutta (SPRK) methods are known to be a good choice in forward simulations of Hamiltonian systems due to their structure-preserving properties. Recent works study the application of SPRK methods to nonlinear and linear-quadratic optimal control problems howing various advantages of these methods compared to standard non-symplectic integration schemes. Now, our focus is on extending the comparison to SPRK and RK methods of higher orders. For linear-quadratic optimal control problems, we consider the discrete-time Riccati feedback as well as the feedforward control implementation. For applications in which computational power or computation time is limited, low sampling rates are of particular interest. Hence we study this case for the n-fold harmonic oscillator

Flaßkamp, Kathrin

Herrmann-Wicklmayr, Markus

PAMM

English

Acronym

Received: 15 June 2021 Accepted: 20 September 2021DOI: 10.1002/pamm.202100076Symplectic Partitioned Runge-Kutta Methods for High-OrderApproximation in Linear-Quadratic Optimal Control of HamiltonianSystemsMarkus Herrmann-Wicklmayr1,∗ and Kathrin Flaßkamp1,∗∗1 Saarland University, Campus, 66123 SaarbrückenSymplectic partitioned Runge-Kutta (SPRK) methods are known to be a good choice in forward simulations of Hamiltoniansystems due to their structure-preserving properties. Recent works study the application of SPRK methods to nonlinear andlinear-quadratic optimal control problems howing various advantages of these methods compared to standard non-symplecticintegration schemes. Now, our focus is on extending the comparison to SPRK and RK methods of higher orders. Forlinear-quadratic optimal control problems, we consider the discrete-time Riccati feedback as well as the feedforward controlimplementation. For applications in which computational power or computation time is limited, low sampling rates are ofparticular interest. Hence we study this case for the n-fold harmonic oscillator.© 2021 The Authors. Proceedings in Applied Mathematics & Mechanics published by Wiley-VCH GmbH.1 IntroductionWe consider the linear-quadratic (LQ) optimal control problem for a linear time-varying (LTV) Hamiltonian system, i.e. thesystem matrix A is Hamiltonian and the state x consists of configurations qi and momenta pi:minuJ (u) = 12∫ T0∥x(t) − xref(t)∥2Q+ ∥u(t)∥2Rdt + 12∥x(T ) − xref(T )∥2PTw.r.t. ẋ(t) = A(t)x(t) +B(t)u(t), x(0) = x0, x(t) = (q1, . . . , qn, p1, . . . , pn)⊺ (t).For this problem, the Ricatti equations can be derived by using the Hamilton function of the problem and applying Pontryaginsmaximum principle. For its discrete-time counterpart, the Lagrangian is derived and the Karush-Kuhn-Tucker conditions areapplied. A sketch of both derivations is shown in Figure 1, where the costate is denoted as θ and the state-costate vector asz ∶= [x, θ]1. The complete derivation of the discrete-time case for SPRK methods, including the undefined quantities, can befound in [1].continuous-time discrete-timeż ==∶ϕCTz³¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹·¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹µ( A 0−Q −A⊺ ) z + ( B0 )²=∶ϕCTuu + ( 0Q)²=∶ϕCTxrefxref zk+1 ==∶ϕz³¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹·¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹µ⎛⎜⎝I + hȞbAsF1 0−hȞb (Qs +A⊺sL2)F1 I − hȞbA⊺sL1⎞⎟⎠ zk+⎛⎜⎝hȞb (AsF2 +Bs)−hȞb (Qs +A⊺sL2)F2⎞⎟⎠´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶=∶ϕUUk + ⎛⎜⎝0hȞb (Qs +A⊺sL2)⎞⎟⎠´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶=∶ϕXrefX refku = u∗ = −R−1B⊺θ Uk = U∗k = Cz+zk+1 +Czzk +CX refX refk , u∗k = −R−1B⊺θkż ==∶ΦCT³¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹·¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹µ( A −BR−1B⊺−Q −A⊺ ) z + ( 0Q )²=∶OCTxref zk+1 ==∶Φ³¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹·¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹µ(I − ϕUCz+)−1⋅ (ϕz + ϕUCz) zk+ (I − ϕUCz+)−1⋅ (ϕUCX ref + ϕX ref)´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶=∶OX refkansatz: θ = Px + r ansatz: θk = Pkxk + rk and αk = [α1k, α2k] ∶= OX refkṖ = − (PA +A⊺P +Q − PBR−1B⊺P) , P (T ) = PT ,ṙ = − (A⊺r −Qxref − PBR−1B⊺r) , r(T ) = −PTxref(T )Pk = (Φ22 − Pk+1Φ12)−1 (−Φ21 + Pk+1Φ11) , PN = PT ,rk = (Φ22 − Pk+1Φ12)−1 (rk+1 − α2k + Pk+1α1k) , rN = −PNxrefNFig. 1: Comparison of the derivation of the solution for the continuous-time and discrete-time case.∗ e-mail markus.herrmannwicklmayr@uni-saarland.de∗∗ e-mail kathrin.flasskamp@uni-saarland.de1 The vertical concatenation of the two matrices X,Y with consistent column dimensions is denoted with [X,Y ] ∶= (X⊺, Y ⊺)⊺.This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution inany medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.PAMM ⋅ Proc. Appl. Math. Mech. 2021;21:1 e202100076. www.gamm-proceedings.com 1 of 2https://doi.org/10.1002/pamm.202100076 © 2021 The Authors. Proceedings in Applied Mathematics & Mechanics published by Wiley-VCH GmbH.2 of 2 Section 20: Dynamics and control2 Numerical Analysis of the Discrete-Time System DynamicsIn this section, we analyse the convergence of an implicit 3-stage, 4th order SPRK Gaussian method (cf. references in [1]) andan explicit 7-stage, 6th order method, given in [2], with bi > 0 for all i = 1, . . . ,7. The non-zero weights are a prerequisite forconstructing the adjoint method characterized by the adjoint Butcher tableau (Ā, b̄ = b, c̄ = c), where Āi,j = (bibj − bjAj,i) /biand (A,b, c) as the original tableau. Following Butcher’s simplifying assumption C(2) [3], one usually sets b2 = 0. Thus, thechosen explicit 6th-order method is exceptional and, to the best of the authors’ knowledge, there has not been found an explicitmethod of higher order satisfying b2 ≠ 0.Each simulation of the equations in the right column of Figure 1 yields sequences of states, costates, and inputs that arecompared with a reference solution which has been obtained numerically but with high accuracy. Our numerical analysis isbased on a Monte-Carlo simulation. Therefore n-fold harmonic oscillators with different size and random physical parameters(resulting from the random number generator seed σ), are generated. For these systems the origin is to be stabilized. For eachMonte-Carlo run depending on the system (n,σ), chosen method r, step size h and state weighting matrix Q = Qcoeff ⋅ Q̃,Qcoeff ∈ R, we compute the maximal error and average them over all systems usedey,max (n,σ, r, h,Qcoeff) = maxk=0,...,N ∥yk − ybvp(k ⋅ h)∥∞ ⇒ ēy,max (r, h,Qcoeff) = (n̂ ⋅ σ̂)−1 ⋅nn̂∑n=n1σσ̂∑σ=σ1 ey,max(. . .),where y ∈ {x, θ, u}. These averaged errors can be plotted logarithmically for each method r . The results are shown exemplarilyfor the input u in Figure 2. In Figure 2(a), it can be seen that for a fixed step size the explicit method outperforms the SPRK(a) Plotted over step size h. (b) Plotted over fct. evals. per (sim.) second.Fig. 2: Errors w.r.t. the optimal input.method when the maximal error is smaller than 10−2. The situation significantly changes when the function evaluations persimulation second are fixed, see Figure 2(b). Now using the SPRK method is advantageous. The same qualitative behaviourcan be observed for the quantities x and θ and, also, the trajectory tracking case.3 ConclusionWe compared a 4th-order SPRK to an explicit 6th-order method for the discretization of a linear-quadratic optimal controlproblem using the discrete Riccati equations derived in [1]. As it is well known from numerical error analysis for numericalintegration, the higher-order method has smaller error norms for h→ 0. However, we showed that it can still be advantageousto use the lower order method: As depicted in Figure 2, the lower-order symplectic method outperforms the other method a) forlarge step sizes and b) when the number of function evaluations play a crucial role.Acknowledgements Open access funding enabled and organized by Projekt DEAL.References[1] M. Herrmann-Wicklmayr and K. Flaßkamp, Approximate LQ Optimal Control Using High-Order Symplectic Partitioned Runge-KuttaMethods, ECC (accepted), (2021).[2] S. Khashin, A Symbolic-Numeric Approach to the Solution of the Butcher Equations, CAMQ, (2009).[3] J. C. Butcher, Numerical Methods for Ordinary Differential Equations, 3rd edition (Wiley, 2016)© 2021 The Authors. Proceedings in Applied Mathematics & Mechanics published by Wiley-VCH GmbH. www.gamm-proceedings.com

Symplectic Partitioned Runge‐Kutta Methods for High‐Order Approximation in Linear‐Quadratic Optimal Control of Hamiltonian Systems

Universaar

https://publikationen.sulb.uni-saarland.de/bitstream/20.500.11880/32520/1/Proc%20Appl%20Math%20%20%20Mech%20-%202021%20-%20Herrmann-Wicklmayr%20-%20Symplectic%20Partitioned%20Runge%e2%80%90Kutta%20Methods%20for%20High%e2%80%90Order%20Approximation.pdf

Symplectic Partitioned Runge‐Kutta Methods for High‐Order Approximation in Linear‐Quadratic Optimal Control of Hamiltonian Systems

Abstract

Similar works

Full text

Available Versions

Acronym

Universaar