In this paper, we consider a general nonlinear control system that is subject to both terminal state and continuous inequality constraints. The continuous inequality constraints must be satisfied at every point in the time horizon—an infinite number of points. Our aim is to design an optimal feedback controller that yields efficient system performance and satisfaction of all constraints. We first formulate this problem as a semi-infinite optimization problem. We then show that, by using a novel exact penalty approach, this semi-infinite optimization problem can be converted into a sequence of nonlinear programming problems, each of which can be solved using standard numerical techniques. We conclude the paper with some convergence results

Lin, Qun

Loxton, Ryan

English

espace@Curtin

Proceedings of the 5th International Conference on Optimization and Control with Applications(OCA2012), Beijing, China, December 4-8, 2012111 OPTIMAL STATE FEEDBACK FORCONSTRAINED NONLINEAR SYSTEMSQun Lin*a, Ryan LoxtonDepartment of Mathematics and StatisticsCurtin University, Perth, Western AustraliaAbstract: In this paper, we consider a general nonlinear control system that is subject to both terminalstate and continuous inequality constraints. The continuous inequality constraints must be satisfied at everypoint in the time horizon—an infinite number of points. Our aim is to design an optimal feedback controllerthat yields efficient system performance and satisfaction of all constraints. We first formulate this problemas a semi-infinite optimization problem. We then show that, by using a novel exact penalty approach, thissemi-infinite optimization problem can be converted into a sequence of nonlinear programming problems,each of which can be solved using standard numerical techniques. We conclude the paper with someconvergence results.Key words: Optimal control; State feedback; Exact penalty function; Nonlinear programming.1 PROBLEM FORMULATIONWe consider nonlinear control systems in the following general form:ẋ(t) = f(t,x(t),u(t)), t ∈ [0, T ], (1.1)x(0) = x0, (1.2)where x(t) ∈ Rn is the state, u(t) ∈ Rr is the control, x0 ∈ Rn is a given initial state, T is a giventerminal time, and f : R× Rn × Rr → Rn is a given continuously differentiable function.System (1.1)-(1.2) is subject to the following terminal state constraints :Ψi(x(T )) = 0, i = 1, . . . , p, (1.3)where Ψi : Rn → R, i = 1, . . . , p are given continuously differentiable functions.In addition, system (1.1)-(1.2) is subject to a set of continuous inequality constraints defined as follows:hj(t,x(t),u(t)) ≤ 0, t ∈ [0, T ], j = 1, . . . , q, (1.4)where hj : R × Rn × Rr → R, j = 1, . . . , q are given continuously differentiable functions. Note thatcontrol bounds can be easily incorporated into (1.4).Our aim is to design an optimal state feedback control for system (1.1)-(1.2). To this end, we assumethat the control takes the following form:u(t) = ϕ(x(t), ζ), t ∈ [0, T ], (1.5)where ζ ∈ Rm is a vector of feedback control parameters and ϕ : Rn × Rm → Rr is a given continuouslydifferentiable function. Typical choices for the feedback controller (1.5) include linear state feedback337338control (in which ϕ is a linear function—see Khalil H. K. (2002)) and PID control (in which ϕ is thesum of linear, integral, and derivative terms—see Li B. (2011)).The feedback control parameters ζk, k = 1, . . . ,m are subject to the following bound constraints:ak ≤ ζk ≤ bk, k = 1, . . . ,m, (1.6)where ak and bk, k = 1, . . . ,m are given constants. Let Γ denote the set of all ζ ∈ Rm satisfying (1.6).Substituting (1.5) into (1.1) givesẋ(t) = f̃(t,x(t), ζ), t ∈ [0, T ], (1.7)wheref̃(t,x(t), ζ) = f(t,x(t),ϕ(x(t), ζ)).Let x(·|ζ) denote the solution of system (1.7) with the initial condition (1.2). Then the terminal con-straints (1.3) becomeΨi(x(T |ζ)) = 0, i = 1, . . . , p. (1.8)Substituting the feedback control (1.5) into the continuous inequality constraints (1.4) givesh̃j(t,x(t|ζ), ζ) ≤ 0, t ∈ [0, T ], j = 1, . . . , q, (1.9)whereh̃j(t,x(t|ζ), ζ) = hj(t,x(t|ζ),ϕ(x(t|ζ), ζ)).Let Λ denote the set of all ζ ∈ Γ satisfying (1.8) and (1.9).We now consider the problem of choosing the feedback control parameters ζk, k = 1, . . . ,m to minimizethe total system cost subject to the constraints (1.8) and (1.9).Problem P Choose ζ ∈ Λ to minimize the cost functionJ(ζ) = Φ(x(T |ζ), ζ) +￿ T0L(t,x(t|ζ), ζ)dt,where Φ : Rn × Rm → R and L : R× Rn × Rm → R are given continuously differentiable functions.Note that (1.9) defines an infinite number of constraints—one for each point in [0, T ]. Hence, Problem Pcan be viewed as a semi-infinite optimization problem. In the next section, we will use a novel exactpenalty approach to approximate Problem P by a nonlinear programming problem.2 AN EXACT PENALTY METHODDefine a constraint violation function on Γ as follows:∆(ζ) =p￿i=1Ψi(x(T |ζ))2 +q￿j=1￿ T0max￿h̃j(t,x(t|ζ), ζ), 0￿2dt.Clearly, ∆(ζ) = 0 if and only if ζ ∈ Λ.Let ￿̄ > 0 be a given constant. We consider the following penalty function defined on Γ× [0, ￿̄]:Gσ(ζ, ￿) =J(ζ), if ￿ = 0, ∆(ζ) = 0,J(ζ) + ￿−α∆(ζ) + σ￿β , if ￿ ∈ (0, ￿̄],∞, if ￿ = 0, ∆(ζ) ￿= 0,(2.1)where ￿ ∈ [0, ￿̄] is a new decision variable, α and β are fixed constants such that 1 ≤ β ≤ α, and σ > 0 isa penalty parameter.In the penalty function (2.1), the last term σ￿β is designed to penalize large values of ￿, while the middleterm ￿−α∆(ζ) is designed to penalize constraint violations. When σ is large, minimizing (2.1) forces ￿to be small, which in turn causes ￿−α to become large, and thus constraint violations are penalized veryseverely. Hence, minimizing the penalty function for large σ will likely lead to feasible points satisfyingconstraints (1.8) and (1.9). On this basis, we can approximate Problem P by the following penaltyproblem.OPTIMAL STATE FEEDBACK FOR CONSTRAINED NONLINEAR SYSTEMS 339Problem Q Choose (ζ, ￿) ∈ Γ× (0, ￿̄] to minimize the penalty functionGσ(ζ, ￿) = J(ζ) + ￿−α∆(ζ) + σ￿β .Problem Q only involves bound constraints and is therefore much easier to solve than Problem P. In thenext section, we will present some convergence results that formally link Problem Q with Problem P.First, however, we discuss how to solve Problem Q.Problem Q can be viewed as a nonlinear programming problem in which the feedback control parame-ters ζk, k = 1, . . . ,m and the new decision variable ￿ need to be chosen to minimize the penalty functionGσ. Numerical algorithms for solving such problems typically use the gradient of the cost function tocompute descent directions that lead to more profitable areas of the feasible region (Luenberger D. G.(2008)). Notice, however, that ζ influences Gσ implicitly through the dynamic system (1.7), and thuscomputing the gradient of Gσ is not straightforward. Nevertheless, the techniques developed by Vincentand Grantham (Vincent (1981)) and Loxton et al. (Loxton R. (2008)) can be used to derive formulae forthe gradient of Gσ. This is described below.First, for each k = 1, . . . ,m, consider the following variational system:φ̇k(t) =∂f̃(t,x(t|ζ), ζ)∂xφk(t) +∂f̃(t,x(t|ζ), ζ)∂ζk, t ∈ [0, T ], (2.2)φk(0) = 0. (2.3)Let φk(·|ζ) denote the solution of the variational system (2.2)-(2.3). We have the following result.Theorem 2.1 For each k = 1, . . . ,m,∂x(t|ζ)∂ζk= φk(t|ζ), t ∈ [0, T ].Proof. First, note that∂∂ζk￿x(0|ζ)￿=∂∂ζk￿x0￿= 0. (2.4)Thus, ∂x(·|ζ)/∂ζk satisfies the initial condition (2.3).Now, by (1.7),x(t|ζ) = x(0|ζ) +￿ t0f̃(s,x(s|ζ), ζ)ds = x0 +￿ t0f̃(s,x(s|ζ), ζ)ds, t ∈ [0, T ]. (2.5)It can be shown that x(t|ζ) is a continuously differentiable function of ζk, k = 1, . . . ,m (Loxton R.(2011)). Hence, by using Leibniz’s rule to differentiate (2.5) with respect to ζk, we obtain∂x(t|ζ)∂ζk=￿ t0￿∂f̃(s,x(s|ζ), ζ)∂x∂x(s|ζ)∂ζk+∂f̃(s,x(s|ζ), ζ)∂ζk￿ds, t ∈ [0, T ], (2.6)where∂f̃(s,x(s|ζ), ζ)∂x=∂f(s,x(s|ζ),ϕ(x(s|ζ), ζ))∂x+∂f(s,x(s|ζ),ϕ(x(s|ζ), ζ))∂u∂ϕ(x(s|ζ), ζ)∂xand∂f̃(s,x(s|ζ), ζ)∂ζk=∂f(s,x(s|ζ),ϕ(x(s|ζ), ζ))∂u∂ϕ(x(s|ζ), ζ)∂ζk.Differentiating (2.6) with respect to time yieldsddt￿∂x(t|ζ)∂ζk￿=∂f̃(t,x(t|ζ), ζ)∂x∂x(t|ζ)∂ζk+∂f̃(t,x(t|ζ), ζ)∂ζk, t ∈ [0, T ]. (2.7)Equations (2.4) and (2.7) show that ∂x(·|ζ)/∂ζk is the solution of the variational system (2.2)-(2.3). Thiscompletes the proof. ✷We are now ready to derive formulae for the gradient of Gσ.340Theorem 2.2 The partial derivatives of Gσ are given by∂Gσ(ζ, ￿)∂ζk=∂J(ζ)∂ζk+ ￿−α∂∆(ζ)∂ζk, k = 1, . . . ,m, (2.8)and∂Gσ(ζ, ￿)∂￿= −α￿−α−1∆(ζ) + βσ￿β−1, (2.9)where∂J(ζ)∂ζk=∂Φ(x(T |ζ), ζ)∂xφk(T |ζ) + ∂Φ(x(T |ζ), ζ)∂ζk+￿ T0￿∂L(t,x(t|ζ), ζ)∂xφk(t|ζ) + ∂L(t,x(t|ζ), ζ)∂ζk￿dt,∂∆(ζ)∂ζk= 2p￿i=1Ψi(x(T |ζ))∂Ψi(x(T |ζ))∂xφk(T |ζ)+ 2q￿j=1￿ T0max￿h̃j(t,x(t|ζ), ζ), 0￿￿∂h̃j(t,x(t|ζ), ζ)∂xφj(t|ζ) + ∂h̃j(t,x(t|ζ), ζ)∂ζk￿dt.Proof. From Theorem 2.1, we have∂J(ζ)∂ζk=∂Φ(x(T |ζ), ζ)∂x∂x(T |ζ)∂ζk+∂Φ(x(t|ζ), ζ)∂ζk+￿ T0￿∂L(t,x(t|ζ), ζ)∂x∂x(t|ζ)∂ζk+L(t,x(t|ζ), ζ)∂ζk￿dt=∂Φ(x(T |ζ), ζ)∂xφk(T |ζ) + ∂Φ(x(T |ζ), ζ)∂ζk+￿ T0￿∂L(t,x(t|ζ), ζ)∂xφk(t|ζ) + L(t,x(t|ζ), ζ)∂ζk￿dt.Similarly,∂∆(ζ)∂ζk= 2p￿i=1Ψi(x(T |ζ))∂Ψi(x(T |ζ))∂xφk(T |ζ)+ 2q￿j=1￿ T0max￿h̃j(t,x(t|ζ), ζ), 0￿￿∂h̃j(t,x(t|ζ), ζ)∂x∂x(t|ζ)∂ζk+∂h̃j(t,x(t|ζ), ζ)∂ζk￿dt= 2p￿i=1Ψi(x(T |ζ))∂Ψi(x(T |ζ))∂xφk(T |ζ)+ 2q￿j=1￿ T0max￿h̃j(t,x(t|ζ), ζ), 0￿￿∂h̃j(t,x(t|ζ), ζ)∂xφk(t|ζ) + ∂h̃j(t,x(t|ζ), ζ)∂ζk￿dt.Equation (2.8) follows immediately from these equations. Equation (2.9) is obtained using standarddifferentiation rules. ✷On the basis of Theorem 2.2, we can compute the gradient of Gσ using the following procedure:(i) Combine the original control system with the variational systems to form an expanded initial valueproblem; (ii) Solve this expanded initial value problem using a numerical integration method; (iii) Sub-stitute the solution of the initial value problem into (2.8) and (2.9). This procedure can be integratedwith a standard gradient-based optimization method—e.g. sequential quadratic programming (NocedalJ. (2006))—to solve Problem Q as a nonlinear programming problem.3 CONVERGENCE RESULTSIn this section, we describe the mathematical theory relating Problem P with Problem Q. We begin withthe following result proved by Lin et al. (Lin Q. (2012)).Theorem 3.1 Let {σl}∞l=1 be an increasing sequence of penalty parameters such that σl → ∞ as l → ∞.Furthermore, let (ζl,∗, ￿l,∗) denote a global solution of Problem Q. Then the sequence {(ζl,∗, ￿l,∗)}∞l=1 hasat least one limit point, and any limit point is a global solution of Problem P.Theorem 3.1 suggests that we can obtain a solution of Problem P by solving Problem Q sequentiallyfor increasing values of the penalty parameter. As mentioned in the previous section, Problem Q isREFERENCES 341essentially a nonlinear programming problem that can be solved using standard numerical optimizationtechniques.One disadvantage of Theorem 3.1 is that it requires the global solution of Problem Q. Problem Q isnon-convex in general, and thus we will usually only be able to solve it locally. Nevertheless, by makingsome mild assumptions, one can show that a local solution of Problem Q converges to a local solution ofProblem P as the penalty parameter increases.We assume that for each feasible point ζ ∈ Λ of Problem P, the following conditions are satisfied:(A1) The vectors ∂Ψi(x(T |ζ))/∂ζ, i = 1, . . . , p are linearly independent (when p ￿= 0).(A2) There exists a vector [η1, . . . , ηm]￿ ∈ Rm and negative real numbers ϑ1 < 0 and ϑ2 < 0 such thatm￿k=1ηk∂Ψi(x(T |ζ))∂ζk= 0, i = 1, . . . , p,m￿k=1ηk￿∂h̃j(t,x(t|ζ), ζ)∂xφk(t|ζ) + ∂h̃j(t,x(t|ζ), ζ)∂ζk￿< ϑ1,t ∈ { s ∈ [0, T ] : h̃j(s,x(s|ζ), ζ) ≥ ϑ2 }, j = 1, . . . , q,ηk￿> 0, if ζk = ak,< 0, if ζk = bk.(A3) There exists a constant L > 0 and a neighbourhood N of ζ such that for each j = 1, . . . , q,max￿h̃j(t,x(t|ζ￿), ζ￿), 0￿2 ≤ L￿ T0max￿h̃j(s,x(s|ζ￿), ζ￿), 0￿2ds, (ζ￿, t) ∈ N × [0, T ].Under Assumptions (A1)-(A3), we have the following result proved by Lin et al. (Lin Q. (2012)).Theorem 3.2 Let {σl}∞l=1 be an increasing sequence of penalty parameters such that σl → ∞ as l → ∞.Furthermore, let (ζl,∗, ￿l,∗) denote a local solution of Problem Q. Suppose that {Gσl(ζl,∗, ￿l,∗)}∞l=1 isbounded. Then there exists a positive integer l￿ such that for each l ≥ l￿, ζl,∗ is a local solution ofProblem P.Theorem 3.2 implies that when the penalty parameter σ is sufficiently large, the values of the feedbackcontrol parameters in a locally optimal solution for Problem Q will also be locally optimal for Problem P.On this basis, we propose the following algorithm for solving Problem P:(1) Choose ζ0 ∈ Γ (initial guess), σ0 > 0 (initial penalty parameter), ρ > 0 (tolerance), and σmax > σ0(upper bound for the penalty parameter).(2) Set ￿̄ → ￿0 and σ0 → σ.(3) Starting with (ζ0, ￿0) as the initial guess, use a nonlinear programming algorithm (e.g. sequentialquadratic programming) to solve Problem Q. Let (ζ∗, ￿∗) denote the local minimizer obtained.(4) If ￿∗ < ρ, then stop: take ζ∗ as a local solution of Problem P. Otherwise, set 10σ → σ and go toStep 5.(5) If σ ≤ σmax, then set (ζ∗, ￿∗) → (ζ0, ￿0) and go to Step 3. Otherwise stop: the algorithm cannotfind a solution of Problem P.ReferencesH. K. Khalil, Nonlinear Systems, 3rd edition, Prentice Hall, New Jersey, USA, 2002.B. Li, K. L. Teo, C. C. Lim, and G. R. Duan, “An optimal PID controller design for nonlinear constrainedoptimal control problems,” Discrete and Continuous Dynamical Systems – Series B, vol. 16, no. 4,pp. 1101-1117, 2011.Q. Lin, R. Loxton, K. L. Teo, Y. H. Wu, and C. Yu, “A new exact penalty method for semi-infiniteprogramming problems,” submitted.R. Loxton, K. L. Teo, and V. Rehbock, “Optimal control problems with multiple characteristic timepoints in the objective and constraints,” Automatica, vol. 44, no. 11, pp. 2923-2929, 2008.342R. Loxton, K. L. Teo, and V. Rehbock, “Robust suboptimal control of nonlinear systems,” AppliedMathematics and Computation, vol. 217, no. 14, pp. 6566-6576, 2011.D. G. Luenberger and Y. Ye, Linear and Nonlinear Programming, 3rd edition, Springer, New York, USA,2008.J. Nocedal and S. J. Wright, Numerical Optimization, 2nd edition, Springer, New York, USA, 2006.K. L. Teo, C. J. Goh, and K. H. Wong, A Unified Computational Approach to Optimal Control Problems,Longman Scientific and Technical, Essex, UK, 1991.T. L. Vincent and W. J. Grantham, Optimality in Parametric Systems, John Wiley, New York, USA,1981.

Optimal state feedback for constrained nonlinear systems

https://espace.curtin.edu.au/bitstream/20.500.11937/25007/2/188627_68944_Lin_Loxton_OCA_2012.pdf

Optimal state feedback for constrained nonlinear systems

Abstract

Similar works

Full text

Available Versions

espace@Curtin