Formal Abstraction of General Stochastic Systems via Noise Partitioning

Verifying the performance of safety-critical, stochastic systems with complex noise distributions is difficult. We introduce a general procedure for the finite abstraction of nonlinear stochastic systems with non-standard (e.g., non-affine, non-symmetric, non-unimodal) noise distributions for verification purposes. The method uses a finite partitioning of the noise domain to construct an interval Markov chain (IMC) abstraction of the system via transition probability intervals. Noise partitioning allows for a general class of distributions and structures, including multiplicative and mixture models, and admits both known and data-driven systems. The partitions required for optimal transition bounds are specified for systems that are monotonic with respect to the noise, and explicit partitions are provided for affine and multiplicative structures. By the soundness of the abstraction procedure, verification on the IMC provides guarantees on the stochastic system against a temporal logic specification. In addition, we present a novel refinement-free algorithm that improves the verification results. Case studies on linear and nonlinear systems with non-Gaussian noise, including a data-driven example, demonstrate the generality and effectiveness of the method without introducing excessive conservatism.Comment: 6 pages, 6 figures, submitted jointly to IEEE Control Systems Letters and 2024 AC

A Decoupling Principle for Simultaneous Localization and Planning Under Uncertainty in Multi-Agent Dynamic Environments

Simultaneous localization and planning for nonlinear stochastic systems under process and measurement uncertainties is a challenging problem. In its most general form, it is formulated as a stochastic optimal control problem in the space of feedback policies. The Hamilton-Jacobi-Bellman equation provides the theoretical solution of the optimal problem; but, as is typical of almost all nonlinear stochastic systems, optimally solving the problem is intractable. Moreover, even if an optimal solution was obtained, it would require centralized control, while multi-agent mobile robotic systems under dynamic environments require decentralized solutions. In this study, we aim for a theoretically sound solution for various modes of this problem, including the single-agent and multi-agent variations with perfect and imperfect state information, where the underlying state, control and observation spaces are continuous with discrete-time models. We introduce a decoupling principle for planning and control of multi-agent nonlinear stochastic systems based on a small noise asymptotics. Through this decoupling principle, under small noise, the design of the real-time feedback law can be decoupled from the off-line design of the nominal trajectory of the system. Further, for a multi-agent problem, the design of the feedback laws for different agents can be decoupled from each other, reducing the centralized problem to a decentralized problem requiring no communication during execution. The resulting solution is quantifiably near-optimal. We establish this result for all the above-mentioned variations, which results in the following variants: Trajectory-optimized Linear Quadratic Regulator (T-LQR), Multi-agent T-LQR (MT-LQR), Trajectory-optimized Linear Quadratic Gaussian (T-LQG), and Multi-agent T-LQG (MT-LQG). The decoupling principle provides the conditions under which a decentralized linear Gaussian system with a quadratic approximation of the cost, obtained by linearization around an optimally designed nominal trajectory can be utilized to control the nonlinear system. The resulting decentralized feedback solution at runtime, being decoupled with respect to the mobile agents, requires no communication between the agents during the execution phase. Moreover, the complexity of the solution vis-a-vis the computation of the nominal trajectory as well as the closed-loop gains is tractable with low polynomial orders of computation. Experimental implementation of the solution shows that the results hold for moderate levels of noise with high probability. Further optimizing the performance of this approach we show how to design a special cost function for the problem with imperfect state measurement that takes advantage of the fact that the estimation covariance of a linear Gaussian system is deterministic and not dependent on the observations. This design, which corresponds in our overall design to “belief space planning”, incorporates the consequently deterministic cost of the stochastic feedback system into the deterministic design of the nominal trajectory to obtain an optimal nominal trajectory with the best estimation performance. Then, it utilizes the T-LQG approach to design an optimal feedback law to track the designed nominal trajectory. This iterative approach can be used to further tune both the open loop as well as the decentralized feedback gain portions of the overall design. We also provide the multi-agent variant of this approach based on the MT-LQG method. Based on the near-optimality guarantees of the decoupling principle and the TLQG approach, we analyze the performance and correctness of a well-known heuristic in robotic path planning. We show that optimizing measures of the observability Gramian as a surrogate for estimation performance may provide irrelevant or misleading trajectories for planning under observation uncertainty. We then consider systems with non-Gaussian perturbations. An alternative heuristic method is proposed that aims for fast planning in belief space under non- Gaussian uncertainty. We provide a special design approach based on particle filters that results in a convex planning problem implemented via a model predictive control strategy in convex environments, and a locally convex problem in non-convex environments. The environment here refers to the complement of the region in Euclidean space that contains the obstacles or “no fly zones”. For non-convex dynamic environments, where the no-go regions change dynamically with time, we design a special form of an obstacle penalty function that incorporates non-convex time-varying constraints into the cost function, so that the decoupling principle still applies to these problems. However, similar to any constrained problem, the quality of the optimal nominal trajectory is dependent on the quality of the solution obtainable for the nonlinear optimization problem. We simulate our algorithms for each of the problems on various challenging situations, including for several nonlinear robotic models and common measurement models. In particular, we consider 2D and 3D dynamic environments for heterogeneous holonomic and non-holonomic robots, and range and bearing sensing models. Future research can potentially extend the results to more general situations including continuous-time models

A Decoupling Principle for Simultaneous Localization and Planning Under Uncertainty in Multi-Agent Dynamic Environments

Simultaneous localization and planning for nonlinear stochastic systems under process and measurement uncertainties is a challenging problem. In its most general form, it is formulated as a stochastic optimal control problem in the space of feedback policies. The Hamilton-Jacobi-Bellman equation provides the theoretical solution of the optimal problem; but, as is typical of almost all nonlinear stochastic systems, optimally solving the problem is intractable. Moreover, even if an optimal solution was obtained, it would require centralized control, while multi-agent mobile robotic systems under dynamic environments require decentralized solutions. In this study, we aim for a theoretically sound solution for various modes of this problem, including the single-agent and multi-agent variations with perfect and imperfect state information, where the underlying state, control and observation spaces are continuous with discrete-time models. We introduce a decoupling principle for planning and control of multi-agent nonlinear stochastic systems based on a small noise asymptotics. Through this decoupling principle, under small noise, the design of the real-time feedback law can be decoupled from the off-line design of the nominal trajectory of the system. Further, for a multi-agent problem, the design of the feedback laws for different agents can be decoupled from each other, reducing the centralized problem to a decentralized problem requiring no communication during execution. The resulting solution is quantifiably near-optimal. We establish this result for all the above-mentioned variations, which results in the following variants: Trajectory-optimized Linear Quadratic Regulator (T-LQR), Multi-agent T-LQR (MT-LQR), Trajectory-optimized Linear Quadratic Gaussian (T-LQG), and Multi-agent T-LQG (MT-LQG). The decoupling principle provides the conditions under which a decentralized linear Gaussian system with a quadratic approximation of the cost, obtained by linearization around an optimally designed nominal trajectory can be utilized to control the nonlinear system. The resulting decentralized feedback solution at runtime, being decoupled with respect to the mobile agents, requires no communication between the agents during the execution phase. Moreover, the complexity of the solution vis-a-vis the computation of the nominal trajectory as well as the closed-loop gains is tractable with low polynomial orders of computation. Experimental implementation of the solution shows that the results hold for moderate levels of noise with high probability. Further optimizing the performance of this approach we show how to design a special cost function for the problem with imperfect state measurement that takes advantage of the fact that the estimation covariance of a linear Gaussian system is deterministic and not dependent on the observations. This design, which corresponds in our overall design to “belief space planning”, incorporates the consequently deterministic cost of the stochastic feedback system into the deterministic design of the nominal trajectory to obtain an optimal nominal trajectory with the best estimation performance. Then, it utilizes the T-LQG approach to design an optimal feedback law to track the designed nominal trajectory. This iterative approach can be used to further tune both the open loop as well as the decentralized feedback gain portions of the overall design. We also provide the multi-agent variant of this approach based on the MT-LQG method. Based on the near-optimality guarantees of the decoupling principle and the TLQG approach, we analyze the performance and correctness of a well-known heuristic in robotic path planning. We show that optimizing measures of the observability Gramian as a surrogate for estimation performance may provide irrelevant or misleading trajectories for planning under observation uncertainty. We then consider systems with non-Gaussian perturbations. An alternative heuristic method is proposed that aims for fast planning in belief space under non- Gaussian uncertainty. We provide a special design approach based on particle filters that results in a convex planning problem implemented via a model predictive control strategy in convex environments, and a locally convex problem in non-convex environments. The environment here refers to the complement of the region in Euclidean space that contains the obstacles or “no fly zones”. For non-convex dynamic environments, where the no-go regions change dynamically with time, we design a special form of an obstacle penalty function that incorporates non-convex time-varying constraints into the cost function, so that the decoupling principle still applies to these problems. However, similar to any constrained problem, the quality of the optimal nominal trajectory is dependent on the quality of the solution obtainable for the nonlinear optimization problem. We simulate our algorithms for each of the problems on various challenging situations, including for several nonlinear robotic models and common measurement models. In particular, we consider 2D and 3D dynamic environments for heterogeneous holonomic and non-holonomic robots, and range and bearing sensing models. Future research can potentially extend the results to more general situations including continuous-time models

Study of the best linear approximation of nonlinear systems with arbitrary inputs

System identification is the art of modelling of a process (physical, biological, etc.) or to predict its behaviour or output when the environment condition or parameter changes. One is modelling the input-output relationship of a system, for example, linking temperature of a greenhouse (output) to the sunlight intensity (input), power of a car engine (output) with fuel injection rate (input). In linear systems, changing an input parameter will result in a proportional increase in the system output. This is not the case in a nonlinear system. Linear system identification has been extensively studied, more so than nonlinear system identification. Since most systems are nonlinear to some extent, there is significant interest in this topic as industrial processes become more and more complex. In a linear dynamical system, knowing the impulse response function of a system will allow one to predict the output given any input. For nonlinear systems this is not the case. If advanced theory is not available, it is possible to approximate a nonlinear system by a linear one. One tool is the Best Linear Approximation (Bla), which is an impulse response function of a linear system that minimises the output differences between its nonlinear counterparts for a given class of input. The Bla is often the starting point for modelling a nonlinear system. There is extensive literature on the Bla obtained from input signals with a Gaussian probability density function (p.d.f.), but there has been very little for other kinds of inputs. A Bla estimated from Gaussian inputs is useful in decoupling the linear dynamics from the nonlinearity, and in initialisation of parameterised models. As Gaussian inputs are not always practical to be introduced as excitations, it is important to investigate the dependence of the Bla on the amplitude distribution in more detail. This thesis studies the behaviour of the Bla with regards to other types of signals, and in particular, binary sequences where a signal takes only two levels. Such an input is valuable in many practical situations, for example where the input actuator is a switch or a valve and hence can only be turned either on or off. While it is known in the literature that the Bla depends on the amplitude distribution of the input, as far as the author is aware, there is a lack of comprehensive theoretical study on this topic. In this thesis, the Blas of discrete-time time-invariant nonlinear systems are studied theoretically for white inputs with an arbitrary amplitude distribution, including Gaussian and binary sequences. In doing so, the thesis offers answers to fundamental questions of interest to system engineers, for example: 1) How the amplitude distribution of the input and the system dynamics affect the Bla? 2) How does one quantify the difference between the Bla obtained from a Gaussian input and that obtained from an arbitrary input? 3) Is the difference (if any) negligible? 4) What can be done in terms of experiment design to minimise such difference? To answer these questions, the theoretical expressions for the Bla have been developed for both Wiener-Hammerstein (Wh) systems and the more general Volterra systems. The theory for the Wh case has been verified by simulation and physical experiments in Chapter 3 and Chapter 6 respectively. It is shown in Chapter 3 that the difference between the Gaussian and non-Gaussian Bla’s depends on the system memory as well as the higher order moments of the non-Gaussian input. To quantify this difference, a measure called the Discrepancy Factor—a measure of relative error, was developed. It has been shown that when the system memory is short, the discrepancy can be as high as 44.4%, which is not negligible. This justifies the need for a method to decrease such discrepancy. One method is to design a random multilevel sequence for Gaussianity with respect to its higher order moments, and this is discussed in Chapter 5. When estimating the Bla even in the absence of environment and measurement noise, the nonlinearity inevitably introduces nonlinear distortions—deviations from the Bla specific to the realisation of input used. This also explains why more than one realisation of input and averaging is required to obtain a good estimate of the Bla. It is observed that with a specific class of pseudorandom binary sequence (Prbs), called the maximum length binary sequence (Mlbs or the m-sequence), the nonlinear distortions appear structured in the time domain. Chapter 4 illustrates a simple and computationally inexpensive method to take advantage this structure to obtain better estimates of the Bla—by replacing mean averaging by median averaging. Lastly, Chapters 7 and 8 document two independent benchmark studies separate from the main theoretical work of the thesis. The benchmark in Chapter 7 is concerned with the modelling of an electrical Wh system proposed in a special session of the 15th International Federation of Automatic Control (Ifac) Symposium on System Identification (Sysid) 2009 (Schoukens, Suykens & Ljung, 2009). Chapter 8 is concerned with the modelling of a ‘hyperfast’ Peltier cooling system first proposed in the U.K. Automatic Control Council (Ukacc) International Conference on Control, 2010 (Control 2010)

An extended orthogonal forward regression algorithm for system identification using entropy

In this paper, a fast identification algorithm for nonlinear dynamic stochastic system identification is presented. The algorithm extends the classical Orthogonal Forward Regression (OFR) algorithm so that instead of using the Error Reduction Ratio (ERR) for term selection, a new optimality criterion —Shannon’s Entropy Power Reduction Ratio(EPRR) is introduced to deal with both Gaussian and non-Gaussian signals. It is shown that the new algorithm is both fast and reliable and examples are provided to illustrate the effectiveness of the new approach

Error-constrained filtering for a class of nonlinear time-varying delay systems with non-gaussian noises

Copyright [2010] IEEE. This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of Brunel University's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to [email protected]. By choosing to view this document, you agree to all provisions of the copyright laws protecting it.In this technical note, the quadratic error-constrained filtering problem is formulated and investigated for discrete time-varying nonlinear systems with state delays and non-Gaussian noises. Both the Lipschitz-like and ellipsoid-bounded nonlinearities are considered. The non-Gaussian noises are assumed to be unknown, bounded, and confined to specified ellipsoidal sets. The aim of the addressed filtering problem is to develop a recursive algorithm based on the semi-definite programme method such that, for the admissible time-delays, nonlinear parameters and external bounded noise disturbances, the quadratic estimation error is not more than a certain optimized upper bound at every time step. The filter parameters are characterized in terms of the solution to a convex optimization problem that can be easily solved by using the semi-definite programme method. A simulation example is exploited to illustrate the effectiveness of the proposed design procedures.This work was supported in part by the Leverhulme Trust of the U.K., the Engineering and Physical Sciences Research Council (EPSRC) of the U.K. under Grant GR/S27658/01, the Royal Society of the U.K., the National Natural Science Foundation of China under Grant 61028008 and Grant 61074016, the Shanghai Natural Science Foundation of China under Grant 10ZR1421200, and the Alexander von Humboldt Foundation of Germany. Recommended by Associate Editor E. Fabre

Nonlinear adaptive control using non-parametric Gaussian Process prior models

Nonparametric Gaussian Process prior models, taken from Bayesian statistics methodology are used to implement a nonlinear adaptive control law. The expected value of a quadratic cost function is minimised, without ignoring the variance of the model predictions. This leads to implicit regularisation of the control signal (caution), and excitation of the system. The controller has dual features, since it is both tracking a reference signal and learning a model of the system from observed responses. The general method and its main features are illustrated on a simulation example

Online-Computation Approach to Optimal Control of Noise-Affected Nonlinear Systems with Continuous State and Control Spaces

© 2007 EUCA.A novel online-computation approach to optimal control of nonlinear, noise-affected systems with continuous state and control spaces is presented. In the proposed algorithm, system noise is explicitly incorporated into the control decision. This leads to superior results compared to state-of-the-art nonlinear controllers that neglect this influence. The solution of an optimal nonlinear controller for a corresponding deterministic system is employed to find a meaningful state space restriction. This restriction is obtained by means of approximate state prediction using the noisy system equation. Within this constrained state space, an optimal closed-loop solution for a finite decision-making horizon (prediction horizon) is determined within an adaptively restricted optimization space. Interleaving stochastic dynamic programming and value function approximation yields a solution to the considered optimal control problem. The enhanced performance of the proposed discrete-time controller is illustrated by means of a scalar example system. Nonlinear model predictive control is applied to address approximate treatment of infinite-horizon problems by the finite-horizon controller