Partially Observed Non-linear Risk-sensitive Optimal Stopping Control for Non-linear Discrete-time Systems

In this paper we introduce and solve the partially observed optimal stopping non-linear risk-sensitive stochastic control problem for discrete-time non-linear systems. The presented results are closely related to previous results for finite horizon partially observed risk-sensitive stochastic control problem. An information state approach is used and a new (three-way) separation principle established that leads to a forward dynamic programming equation and a backward dynamic programming inequality equation (both infinite dimensional). A verification theorem is given that establishes the optimal control and optimal stopping time. The risk-neutral optimal stopping stochastic control problem is also discussed

Approximate dynamic programming based solutions for fixed-final-time optimal control and optimal switching

Optimal solutions with neural networks (NN) based on an approximate dynamic programming (ADP) framework for new classes of engineering and non-engineering problems and associated difficulties and challenges are investigated in this dissertation. In the enclosed eight papers, the ADP framework is utilized for solving fixed-final-time problems (also called terminal control problems) and problems with switching nature. An ADP based algorithm is proposed in Paper 1 for solving fixed-final-time problems with soft terminal constraint, in which, a single neural network with a single set of weights is utilized. Paper 2 investigates fixed-final-time problems with hard terminal constraints. The optimality analysis of the ADP based algorithm for fixed-final-time problems is the subject of Paper 3, in which, it is shown that the proposed algorithm leads to the global optimal solution providing certain conditions hold. Afterwards, the developments in Papers 1 to 3 are used to tackle a more challenging class of problems, namely, optimal control of switching systems. This class of problems is divided into problems with fixed mode sequence (Papers 4 and 5) and problems with free mode sequence (Papers 6 and 7). Each of these two classes is further divided into problems with autonomous subsystems (Papers 4 and 6) and problems with controlled subsystems (Papers 5 and 7). Different ADP-based algorithms are developed and proofs of convergence of the proposed iterative algorithms are presented. Moreover, an extension to the developments is provided for online learning of the optimal switching solution for problems with modeling uncertainty in Paper 8. Each of the theoretical developments is numerically analyzed using different real-world or benchmark problems --Abstract, page v

Optimal Switching Synthesis for Jump Linear Systems with Gaussian initial state uncertainty

This paper provides a method to design an optimal switching sequence for jump linear systems with given Gaussian initial state uncertainty. In the practical perspective, the initial state contains some uncertainties that come from measurement errors or sensor inaccuracies and we assume that the type of this uncertainty has the form of Gaussian distribution. In order to cope with Gaussian initial state uncertainty and to measure the system performance, Wasserstein metric that defines the distance between probability density functions is used. Combining with the receding horizon framework, an optimal switching sequence for jump linear systems can be obtained by minimizing the objective function that is expressed in terms of Wasserstein distance. The proposed optimal switching synthesis also guarantees the mean square stability for jump linear systems. The validations of the proposed methods are verified by examples.Comment: ASME Dynamic Systems and Control Conference (DSCC), 201

On Weak Topology for Optimal Control of Switched Nonlinear Systems

Optimal control of switched systems is challenging due to the discrete nature of the switching control input. The embedding-based approach addresses this challenge by solving a corresponding relaxed optimal control problem with only continuous inputs, and then projecting the relaxed solution back to obtain the optimal switching solution of the original problem. This paper presents a novel idea that views the embedding-based approach as a change of topology over the optimization space, resulting in a general procedure to construct a switched optimal control algorithm with guaranteed convergence to a local optimizer. Our result provides a unified topology based framework for the analysis and design of various embedding-based algorithms in solving the switched optimal control problem and includes many existing methods as special cases

Hybrid control for low-regular nonlinear systems: application to an embedded control for an electric vehicle

This note presents an embedded automatic control strategy for a low consumption vehicle equipped with an "on/off" engine. The main difficulties are the hybrid nature of the dynamics, the non smoothness of the dynamics of each mode, the uncertain environment, the fast changing dynamics, and low cost/ low consumption constraints for the control device. Human drivers of such vehicles frequently use an oscillating strategy, letting the velocity evolve between fixed lower and upper bounds. We present a general justification of this very simple and efficient strategy, that happens to be optimal for autonomous dynamics, robust and easily adaptable for real-time control strategy. Effective implementation in a competition prototype involved in low-consumption races shows that automatic velocity control achieves performances comparable with the results of trained human drivers. Major advantages of automatic control are improved robustness and safety. The total average power consumption for the control device is less than 10 mW

Optimal Switching for Hybrid Semilinear Evolutions

We consider the optimization of a dynamical system by switching at discrete time points between abstract evolution equations composed by nonlinearly perturbed strongly continuous semigroups, nonlinear state reset maps at mode transition times and Lagrange-type cost functions including switching costs. In particular, for a fixed sequence of modes, we derive necessary optimality conditions using an adjoint equation based representation for the gradient of the costs with respect to the switching times. For optimization with respect to the mode sequence, we discuss a mode-insertion gradient. The theory unifies and generalizes similar approaches for evolutions governed by ordinary and delay differential equations. More importantly, it also applies to systems governed by semilinear partial differential equations including switching the principle part. Examples from each of these system classes are discussed