Numerical solution of optimal control problems with constant control delays

We investigate a class of optimal control problems that exhibit constant exogenously given delays in the control in the equation of motion of the differential states. Therefore, we formulate an exemplary optimal control problem with one stock and one control variable and review some analytic properties of an optimal solution. However, analytical considerations are quite limited in case of delayed optimal control problems. In order to overcome these limits, we reformulate the problem and apply direct numerical methods to calculate approximate solutions that give a better understanding of this class of optimization problems. In particular, we present two possibilities to reformulate the delayed optimal control problem into an instantaneous optimal control problem and show how these can be solved numerically with a state-of-the-art direct method by applying Bock’s direct multiple shooting algorithm. We further demonstrate the strength of our approach by two economic examples.delayed differential equations, delayed optimal control, numerical optimization, time-to-build

Numerical Solution of Optimal Control Problems with Constant Control Delays

We investigate a class of optimal control problems that exhibit constant exogenously given delays in the control in the equation of motion of the differential states. Therefore, we formulate an exemplary optimal control problem with one stock and one control variable and review some analytic properties of an optimal solution. However, analytical considerations are quite limited in case of delayed optimal control problems. In order to overcome these limits, we reformulate the problem and apply direct numerical methods to calculate approximate solutions that give a better understanding of this class of optimization problems. In particular, we present two possibilities to reformulate the delayed optimal control problem into an instantaneous optimal control problem and show how these can be solved numerically with a state-of-the-art direct method by applying Bock's direct multiple shooting algorithm. We further demonstrate the strength of our approach by two economic example

Global Optimal Control Using Direct Multiple Shooting

The goal of this thesis is the development of a novel and efficient algorithm to determine the global optimum of an optimal control problem. In contrast to previous methods, the approach presented here is based on the direct multiple shooting method for discretizing the optimal control problem, which results in a significant increase of efficiency. To relax the discretized optimal control problems, the so-called alpha-branch-and-bound method in combination with validated integration is used. For the direct comparison of the direct single-shooting-based relaxations with the direct multipleshooting-based algorithm, several theoretical results are proven that build the basis for the efficiency increase of the novel method. A specialized branching strategy takes care that the additionally introduced variables due to the multiple shooting approach do not increase the size of the resulting branch-and-bound tree. An adaptive scaling technique of the commonly used Gershgorin method to estimate the eigenvalues of interval matrices leads to optimal relaxations and therefore leads to a general improvement of the alpha-branch-and-bound relaxations in a single shooting and a multiple shooting framework, as well as for the corresponding relaxations of non-dynamic nonlinear problems. To further improve the computational time, suggestions regarding the necessary second-order interval sensitivities are presented in this thesis, as well as a heuristic that relaxes on a subspace only. The novel algorithm, as well as the single-shooting-based alternative for a direct comparison, are implemented in a newly developed software package called GloOptCon. The new method is used to globally solve both a number of benchmark problems from the literature, and so far in the context of global optimal control still little considered applications. The additional problems pose new challenges either due to their size or due to having its origin in mixed integer optimal control with an integer-valued time-dependent control variable. The theoretically proven increase of efficiency is validated by the numerical results. Compared to the previous approach from the literature, the number of iterations for typical problems is more than halved, meanwhile the computation time is reduced by almost an order of magnitude. This in turn allows the global solution of significantly larger optimal control problems

A Unified Perspective on Multiple Shooting In Differential Dynamic Programming

Differential Dynamic Programming (DDP) is an efficient computational tool for solving nonlinear optimal control problems. It was originally designed as a single shooting method and thus is sensitive to the initial guess supplied. This work considers the extension of DDP to multiple shooting (MS), improving its robustness to initial guesses. A novel derivation is proposed that accounts for the defect between shooting segments during the DDP backward pass, while still maintaining quadratic convergence locally. The derivation enables unifying multiple previous MS algorithms, and opens the door to many smaller algorithmic improvements. A penalty method is introduced to strategically control the step size, further improving the convergence performance. An adaptive merit function and a more reliable acceptance condition are employed for globalization. The effects of these improvements are benchmarked for trajectory optimization with a quadrotor, an acrobot, and a manipulator. MS-DDP is also demonstrated for use in Model Predictive Control (MPC) for dynamic jumping with a quadruped robot, showing its benefits over a single shooting approach

An intrusive polynomial algebra multiple shooting approach to the solution of optimal control problems

This paper proposes an approach to the solution of optimal control problems under uncertainty, that extends the classical direct multiple shooting transcription to account for random variables defined on extended sets. The proposed approach employs a Generalised Intrusive Polynomial Expansion to model and propagate uncertainty. The development of a generalised framework for a direct multiple shooting transcription of the optimal control problem starts with the discretisation of the time domain in sub-segments. At the beginning of each segment, the state spatial distribution is modelled with a multivariate polynomial and then propagated to the sub-interval final time. Continuity conditions are implicitly imposed at the boundary of two adjacent segments, a critical operation because it requires the continuity of two extended sets. The Intrusive Polynomial Algebra aNd Multiple shooting Approach (IPANeMA) developed in this paper can handle optimal control problems under a wide range of uncertainty models, e.g. nonparametric, expensive to sample, and imprecise probability distributions. In this paper, the approach is applied to the design of a low-thrust trajectory to a Near-Earth Object with uncertain initial conditions

Fast Direct Multiple Shooting Algorithms for Optimal Robot Control

International audienceIn this overview paper, we first survey numerical approaches to solve nonlinear optimal control problems, and second, we present our most recent algorithmic developments for real-time optimization in nonlinear model predictive control. In the survey part, we discuss three direct optimal control approaches in detail: (i) single shooting, (ii) collocation, and (iii) multiple shooting, and we specify why we believe the direct multiple shooting method to be the method of choice for nonlinear optimal control problems in robotics. We couple it with an efficient robot model generator and show the performance of the algorithm at the example of a five link robot arm. In the real-time optimization part, we outline the idea of nonlinear model predictive control and the real-time challenge it poses to numerical optimization. As one solution approach, we discuss the real-time iteration scheme

Parallel Shooting Sequential Quadratic Programming for Nonlinear MPC Problems

In this paper, we propose a parallel shooting algorithm for solving nonlinear model predictive control problems using sequential quadratic programming. This algorithm is built on a two-phase approach where we first test and assess sequential convergence over many initial trajectories in parallel. However, if none converge, the algorithm starts varying the Newton step size in parallel instead. Through this parallel shooting approach, it is expected that the number of iterations to converge to an optimal solution can be decreased. Furthermore, the algorithm can be further expanded and accelerated by implementing it on GPUs. We illustrate the effectiveness of the proposed Parallel Shooting Sequential Quadratic Programming (PS-SQP) method in some benchmark examples for nonlinear model predictive control. The developed PS-SQP parallel solver converges faster on average and especially when significant nonlinear behaviour is excited in the NMPC horizon.Comment: 7 pages, 6 figures, submitted and accepted for the 7th IEEE Conference on Control Technology and Applications (CCTA) 202

GoPRONTO: a Feedback-based Framework for Nonlinear Optimal Control

In this paper we We propose GoPRONTO, a first-order, feedback-based approach to solve nonlinear discrete-time optimal control problems. This method is a generalized first-order framework based on incorporating the original dynamics into a closed-loop system. By exploiting this feedback-based shooting, we are able to reinterpret the optimal control problem as the minimization of a cost function, depending on a state-input curve, whose gradient can be computed by resorting to a suitable costate equation. This convenient reformulation gives room for a collection of accelerated numerical optimal control schemes. To corroborate the theoretical results, numerical simulations on the optimal control of a train of inverted pendulum-on-cart systems are shown.Comment: 27 pages, 8 figure

A Hybrid Direct-Indirect Approach for Solving the Singular Optimal Control Problems of Finite and Infinite Order

This paper presents a hybrid approach to solve singular optimal control problems. It combines the direct Euler method with a modified indirect shooting method. The presented method circumvents the main difficulties and drawbacks of both the direct and indirect methods, when applied to the singular optimal control problems. This method does not require a priori knowledge of the switching structure of the solution and it can be applied to finite or infinite order singular optimal control problems. It provides not only an approximate optimal solution for the problem but, remarkably, it also produces the switching times. We illustrate the features of this new approach treating numerically through two optimal control problems, one of finite order and the other with infinite order