An Online Method for the Data Driven Stochastic Optimal Control Problem with Unknown Model Parameters

In this work, an efficient sample-wise data driven control solver will be developed to solve the stochastic optimal control problem with unknown model parameters. A direct filter method will be applied as an online parameter estimation method that dynamically estimates the target model parameters upon receiving the data, and a sample-wise optimal control solver will be provided to efficiently search for the optimal control. Then, an effective overarching algorithm will be introduced to combine the parameter estimator and the optimal control solver. Numerical experiments will be carried out to demonstrate the effectiveness and the efficiency of the sample-wise data driven control method

Sparse representations and quadratic approximations in path integral techniques for stochastic response analysis of diverse systems/structures

Uncertainty propagation in engineering mechanics and dynamics is a highly challenging problem that requires development of analytical/numerical techniques for determining the stochastic response of complex engineering systems. In this regard, although Monte Carlo simulation (MCS) has been the most versatile technique for addressing the above problem, it can become computationally daunting when faced with high-dimensional systems or with computing very low probability events. Thus, there is a demand for pursuing more computationally efficient methodologies. Recently, a Wiener path integral (WPI) technique, whose origins can be found in theoretical physics, has been developed in the field of engineering dynamics for determining the response transition probability density function (PDF) of nonlinear oscillators subject to non-white, non-Gaussian and non-stationary excitation processes. In the present work, the Wiener path integral technique is enhanced, extended and generalized with respect to three main aspects; namely, versatility, computational efficiency and accuracy. Specifically, the need for increasingly sophisticated modeling of excitations has led recently to the utilization of fractional calculus, which can be construed as a generalization of classical calculus. Motivated by the above developments, the WPI technique is extended herein to account for stochastic excitations modeled via fractional-order filters. To this aim, relying on a variational formulation and on the most probable path approximation yields a deterministic fractional boundary value problem to be solved numerically for obtaining the oscillator joint response PDF. Further, appropriate multi-dimensional bases are constructed for approximating, in a computationally efficient manner, the non-stationary joint response PDF. In this regard, two distinct approaches are pursued. The first employs expansions based on Kronecker products of bases (e.g., wavelets), while the second utilizes representations based on positive definite functions. Next, the localization capabilities of the WPI technique are exploited for determining PDF points in the joint space-time domain to be used for evaluating the expansion coefficients at a relatively low computational cost. Subsequently, compressive sampling procedures are employed in conjunction with group sparsity concepts and appropriate optimization algorithms for decreasing even further the associated computational cost. It is shown that the herein developed enhancement renders the technique capable of treating readily relatively high-dimensional stochastic systems. More importantly, it is shown that this enhancement in computational efficiency becomes more prevalent as the number of stochastic dimensions increases; thus, rendering the herein proposed sparse representation approach indispensable, especially for high-dimensional systems. Next, a quadratic approximation of the WPI is developed for enhancing the accuracy degree of the technique. Concisely, following a functional series expansion, higher-order terms are accounted for, which is equivalent to considering not only the most probable path but also fluctuations around it. These fluctuations are incorporated into a state-dependent factor by which the exponential part of each PDF value is multiplied. This localization of the state-dependent factor yields superior accuracy as compared to the standard most probable path WPI approximation where the factor is constant and state-invariant. An additional advantage relates to efficient structural reliability assessment, and in particular, to direct estimation of low probability events (e.g., failure probabilities), without possessing the complete transition PDF. Overall, the developments in this thesis render the WPI technique a potent tool for determining, in a reliable manner and with a minimal computational cost, the stochastic response of nonlinear oscillators subject to an extended range of excitation processes. Several numerical examples, pertaining to both nonlinear dynamical systems subject to external excitations and to a special class of engineering mechanics problems with stochastic media properties, are considered for demonstrating the reliability of the developed techniques. In all cases, the degree of accuracy and the computational efficiency exhibited are assessed by comparisons with pertinent MCS data

Controlled particle systems for nonlinear filtering and global optimization

This thesis is concerned with the development and applications of controlled interacting particle systems for nonlinear filtering and global optimization problems. These problems are important in a number of engineering domains. In nonlinear filtering, there is a growing interest to develop geometric approaches for systems that evolve on matrix Lie groups. Examples include the problem of attitude estimation and motion tracking in aerospace engineering, robotics and computer vision. In global optimization, the challenges typically arise from the presence of a large number of local minimizers as well as the computational scalability of the solution. Gradient-free algorithms are attractive because in many practical situations, evaluating the gradient of the objective function may be computationally prohibitive. The thesis comprises two parts that are devoted to theory and applications, respectively. The theoretical part consists of three chapters that describe methods and algorithms for nonlinear filtering, global optimization, and numerical solutions of the Poisson equation that arise in both filtering and optimization. For the nonlinear filtering problem, the main contribution is to extend the feedback particle filter (FPF) algorithm to connected matrix Lie groups. In its general form, the FPF is shown to provide an intrinsic coordinate-free description of the filter that automatically satisfies the manifold constraint. The properties of the original (Euclidean) FPF, especially the gain-times-error feedback structure, are preserved in the generalization. For the global optimization problem, a controlled particle filter algorithm is introduced to numerically approximate a solution of the global optimization problem. The theoretical significance of this work comes from its variational aspects: (i) the proposed particle filter is a controlled interacting particle system where the control input represents the solution of a mean-field type optimal control problem; and (ii) the associated density transport is shown to be a gradient flow (steepest descent) for the optimal value function, with respect to the Kullback--Leibler divergence. For both the nonlinear filtering and optimization problems, the numerical implementation of the proposed algorithms require a solution of a Poisson equation. Two numerical algorithms are described for this purpose. In the Galerkin scheme, the gain function is approximated using a set of pre-defined basis functions; In the kernel-based scheme, a numerical solution is obtained by solving a certain fixed-point equation. Well-posedness results for the Poisson equation are also discussed. The second part of the thesis contains applications of the proposed algorithms to specific nonlinear filtering and optimization problems. The FPF is applied to the problem of attitude estimation - a nonlinear filtering problem on the Lie group SO(3). The formulae of the filter are described using both the rotation matrix and the quaternion coordinates. A comparison is provided between FPF and the several popular attitude filters including the multiplicative EKF, the invariant EKF, the unscented Kalman filter, the invariant ensemble Kalman filter and the bootstrap particle filter. Numerical simulations are presented to illustrate the comparison. As a practical application, experimental results for a motion tracking problem are presented. The objective is to estimate the attitude of a wrist-worn motion sensor based on the motion of the arm. In the presence of motion, considered here as the swinging motion of the arm, the observability of the sensor attitude is shown to improve. The estimation problem is mathematically formulated as a nonlinear filtering problem on the product Lie group SO(3)XSO(2), and experimental results are described using data from the gyroscope and the accelerometer installed on the sensor. For the global optimization problem, the proposed controlled particle filter is compared with several model-based algorithms that also employ probabilistic models to inform the search of the global minimizer. Examples of the model-based algorithms include the model reference adaptive search, the cross entropy, the model-based evolutionary optimization, and two algorithms based on bootstrap particle filtering. Performance comparisons are provided between the control-based and the sampling-based implementation. Results of Monte-Carlo simulations are described for several benchmark optimization problems

Uncertainty modelling and computational aspects of data association

A novel solution to the smoothing problem for multi-object dynamical systems is proposed and evaluated. The systems of interest contain an unknown and varying number of dynamical objects that are partially observed under noisy and corrupted observations. In order to account for the lack of information about the different aspects of this type of complex system, an alternative representation of uncertainty based on possibility theory is considered. It is shown how analogues of usual concepts such as Markov chains and hidden Markov models (HMMs) can be introduced in this context. In particular, the considered statistical model for multiple dynamical objects can be formulated as a hierarchical model consisting of conditionally independent HMMs. This structure is leveraged to propose an efficient method in the context of Markov chain Monte Carlo (MCMC) by relying on an approximate solution to the corresponding filtering problem, in a similar fashion to particle MCMC. This approach is shown to outperform existing algorithms in a range of scenarios

Design and Analysis of Stochastic Dynamical Systems with Fokker-Planck Equation

This dissertation addresses design and analysis aspects of stochastic dynamical systems using Fokker-Planck equation (FPE). A new numerical methodology based on the partition of unity meshless paradigm is developed to tackle the greatest hurdle in successful numerical solution of FPE, namely the curse of dimensionality. A local variational form of the Fokker-Planck operator is developed with provision for h- and p- refinement. The resulting high dimensional weak form integrals are evaluated using quasi Monte-Carlo techniques. Spectral analysis of the discretized Fokker- Planck operator, followed by spurious mode rejection is employed to construct a new semi-analytical algorithm to obtain near real-time approximations of transient FPE response of high dimensional nonlinear dynamical systems in terms of a reduced subset of admissible modes. Numerical evidence is provided showing that the curse of dimensionality associated with FPE is broken by the proposed technique, while providing problem size reduction of several orders of magnitude. In addition, a simple modification of norm in the variational formulation is shown to improve quality of approximation significantly while keeping the problem size fixed. Norm modification is also employed as part of a recursive methodology for tracking the optimal finite domain to solve FPE numerically. The basic tools developed to solve FPE are applied to solving problems in nonlinear stochastic optimal control and nonlinear filtering. A policy iteration algorithm for stochastic dynamical systems is implemented in which successive approximations of a forced backward Kolmogorov equation (BKE) is shown to converge to the solution of the corresponding Hamilton Jacobi Bellman (HJB) equation. Several examples, including a four-state missile autopilot design for pitch control, are considered. Application of the FPE solver to nonlinear filtering is considered with special emphasis on situations involving long durations of propagation in between measurement updates, which is implemented as a weak form of the Bayes rule. A nonlinear filter is formulated that provides complete probabilistic state information conditioned on measurements. Examples with long propagation times are considered to demonstrate benefits of using the FPE based approach to filtering

New Algorithms for Uncertainty Quantification and Nonlinear Estimation of Stochastic Dynamical Systems

Recently there has been growing interest to characterize and reduce uncertainty in stochastic dynamical systems. This drive arises out of need to manage uncertainty in complex, high dimensional physical systems. Traditional techniques of uncertainty quantification (UQ) use local linearization of dynamics and assumes Gaussian probability evolution. But several difficulties arise when these UQ models are applied to real world problems, which, generally are nonlinear in nature. Hence, to improve performance, robust algorithms, which can work efficiently in a nonlinear non-Gaussian setting are desired. The main focus of this dissertation is to develop UQ algorithms for nonlinear systems, where uncertainty evolves in a non-Gaussian manner. The algorithms developed are then applied to state estimation of real-world systems. The first part of the dissertation focuses on using polynomial chaos (PC) for uncertainty propagation, and then achieving the estimation task by the use of higher order moment updates and Bayes rule. The second part mainly deals with Frobenius-Perron (FP) operator theory, how it can be used to propagate uncertainty in dynamical systems, and then using it to estimate states by the use of Bayesian update. Finally, a method to represent the process noise in a stochastic dynamical system using a nite term Karhunen-Loeve (KL) expansion is proposed. The uncertainty in the resulting approximated system is propagated using FP operator. The performance of the PC based estimation algorithms were compared with extended Kalman filter (EKF) and unscented Kalman filter (UKF), and the FP operator based techniques were compared with particle filters, when applied to a duffing oscillator system and hypersonic reentry of a vehicle in the atmosphere of Mars. It was found that the accuracy of the PC based estimators is higher than EKF or UKF and the FP operator based estimators were computationally superior to the particle filtering algorithms