Extremum Seeking-based Iterative Learning Model Predictive Control (ESILC-MPC)

In this paper, we study a tracking control problem for linear time-invariant systems, with model parametric uncertainties, under input and states constraints. We apply the idea of modular design introduced in Benosman et al. 2014, to solve this problem in the model predictive control (MPC) framework. We propose to design an MPC with input-to-state stability (ISS) guarantee, and complement it with an extremum seeking (ES) algorithm to iteratively learn the model uncertainties. The obtained MPC algorithms can be classified as iterative learning control (ILC)-MPC

Sample Efficient Path Integral Control under Uncertainty

We present a data-driven optimal control framework that can be viewed as a generalization of the path integral (PI) control approach. We find iterative feedback control laws without parameterization based on probabilistic representation of learned dynamics model. The proposed algorithm operates in a forward-backward manner which differentiate from other PI-related methods that perform forward sampling to find optimal controls. Our method uses significantly less samples to find optimal controls compared to other approaches within the PI control family that relies on extensive sampling from given dynamics models or trials on physical systems in model-free fashions. In addition, the learned controllers can be generalized to new tasks without re-sampling based on the compositionality theory for the linearly-solvable optimal control framework. We provide experimental results on three different systems and comparisons with state-of-the-art model-based methods to demonstrate the efficiency and generalizability of the proposed framework

Reinforcement Learning for Batch Bioprocess Optimization

Bioprocesses have received a lot of attention to produce clean and sustainable alternatives to fossil-based materials. However, they are generally difficult to optimize due to their unsteady-state operation modes and stochastic behaviours. Furthermore, biological systems are highly complex, therefore plant-model mismatch is often present. To address the aforementioned challenges we propose a Reinforcement learning based optimization strategy for batch processes. In this work, we applied the Policy Gradient method from batch-to-batch to update a control policy parametrized by a recurrent neural network. We assume that a preliminary process model is available, which is exploited to obtain a preliminary optimal control policy. Subsequently, this policy is updatedbased on measurements from thetrueplant. The capabilities of our proposed approach were tested on three case studies (one of which is nonsmooth) using a more complex process model for thetruesystemembedded with adequate process disturbance. Lastly, we discussed the advantages and disadvantages of this strategy compared against current existing approaches such as nonlinear model predictive control

Stochastic trajectory optimization for mechanical systems with parametric uncertainties

In this paper we develop a novel, discrete-time optimal control framework for mechanical systems with uncertain model parameters. We consider finite-horizon problems where the performance index depends on the statistical moments of the stochastic system. Our approach constitutes an extension of the original Differential Dynamic Programming method and handles uncertainty through generalized Polynomial Chaos (gPC) theory. The developed iterative scheme is capable of controlling the probabilistic evolution of the dynamic system and can be used in planning and control. Moreover, its scalable and fast converging nature play a key role when dealing with complex, high-dimensional problems. Based on Lagrangian mechanics principles, we also prove that Variational Integrators can be designed to properly propagate and linearize the gPC representations of stochastic, forced mechanical systems in discrete time. We utilize this benefit to further improve the efficiency of our trajectory-optimization methodology. Numerical simulations are included to validate the applicability of our approach

Modeling and identification of uncertain-input systems

In this work, we present a new class of models, called uncertain-input models, that allows us to treat system-identification problems in which a linear system is subject to a partially unknown input signal. To encode prior information about the input or the linear system, we use Gaussian-process models. We estimate the model from data using the empirical Bayes approach: the input and the impulse responses of the linear system are estimated using the posterior means of the Gaussian-process models given the data, and the hyperparameters that characterize the Gaussian-process models are estimated from the marginal likelihood of the data. We propose an iterative algorithm to find the hyperparameters that relies on the EM method and results in simple update steps. In the most general formulation, neither the marginal likelihood nor the posterior distribution of the unknowns is tractable. Therefore, we propose two approximation approaches, one based on Markov-chain Monte Carlo techniques and one based on variational Bayes approximation. We also show special model structures for which the distributions are treatable exactly. Through numerical simulations, we study the application of the uncertain-input model to the identification of Hammerstein systems and cascaded linear systems. As part of the contribution of the paper, we show that this model structure encompasses many classical problems in system identification such as classical PEM, Hammerstein models, errors-in-variables problems, blind system identification, and cascaded linear systems. This allows us to build a systematic procedure to apply the algorithms proposed in this work to a wide class of classical problems.Comment: 27 Pages, submitted to Automatic

New methods for the estimation of Takagi-Sugeno model based extended Kalman filter and its applications to optimal control for nonlinear systems

This paper describes new approaches to improve the local and global approximation (matching) and modeling capability of Takagi–Sugeno (T-S) fuzzy model. The main aim is obtaining high function approximation accuracy and fast convergence. The main problem encountered is that T-S identification method cannot be applied when the membership functions are overlapped by pairs. This restricts the application of the T-S method because this type of membership function has been widely used during the last 2 decades in the stability, controller design of fuzzy systems and is popular in industrial control applications. The approach developed here can be considered as a generalized version of T-S identification method with optimized performance in approximating nonlinear functions. We propose a noniterative method through weighting of parameters approach and an iterative algorithm by applying the extended Kalman filter, based on the same idea of parameters’ weighting. We show that the Kalman filter is an effective tool in the identification of T-S fuzzy model. A fuzzy controller based linear quadratic regulator is proposed in order to show the effectiveness of the estimation method developed here in control applications. An illustrative example of an inverted pendulum is chosen to evaluate the robustness and remarkable performance of the proposed method locally and globally in comparison with the original T-S model. Simulation results indicate the potential, simplicity, and generality of the algorithm. An illustrative example is chosen to evaluate the robustness. In this paper, we prove that these algorithms converge very fast, thereby making them very practical to use

Active Sampling for Constrained Simulation-based Verification of Uncertain Nonlinear Systems

Increasingly demanding performance requirements for dynamical systems motivates the adoption of nonlinear and adaptive control techniques. One challenge is the nonlinearity of the resulting closed-loop system complicates verification that the system does satisfy the requirements at all possible operating conditions. This paper presents a data-driven procedure for efficient simulation-based, statistical verification without the reliance upon exhaustive simulations. In contrast to previous work, this approach introduces a method for online estimation of prediction accuracy without the use of external validation sets. This work also develops a novel active sampling algorithm that iteratively selects additional training points in order to maximize the accuracy of the predictions while still limited to a sample budget. Three case studies demonstrate the utility of the new approach and the results show up to a 50% improvement over state-of-the-art techniques.Comment: 8 pages, submitted to ACC 201

Optimization under Uncertainty in the Era of Big Data and Deep Learning: When Machine Learning Meets Mathematical Programming

This paper reviews recent advances in the field of optimization under uncertainty via a modern data lens, highlights key research challenges and promise of data-driven optimization that organically integrates machine learning and mathematical programming for decision-making under uncertainty, and identifies potential research opportunities. A brief review of classical mathematical programming techniques for hedging against uncertainty is first presented, along with their wide spectrum of applications in Process Systems Engineering. A comprehensive review and classification of the relevant publications on data-driven distributionally robust optimization, data-driven chance constrained program, data-driven robust optimization, and data-driven scenario-based optimization is then presented. This paper also identifies fertile avenues for future research that focuses on a closed-loop data-driven optimization framework, which allows the feedback from mathematical programming to machine learning, as well as scenario-based optimization leveraging the power of deep learning techniques. Perspectives on online learning-based data-driven multistage optimization with a learning-while-optimizing scheme is presented

Fractional-order Generalized Principle of Self-Support (FOG PSS) in Control Systems Design

This paper reviews research that studies the principle of self-support (PSS) in some control systems and proposes a fractional-order generalized PSS framework for the first time. The existing PSS approach focuses on practical tracking problem of integer-order systems including robotic dynamics, high precision linear motor system, multi-axis high precision positioning system with unmeasurable variables, imprecise sensor information, uncertain parameters and external disturbances. More generally, by formulating the fractional PSS concept as a new generalized framework, we will focus in the possible fields on the fractional-order control problems such as practical tracking, $\lambda$-tracking, etc. of robot systems, multiple mobile agents, discrete dynamical systems, time delay systems and other uncertain nonlinear systems. Finally, the practical tracking of a first-order uncertain model of automobile is considered as a simple example to demonstrate the efficiency of the fractional-order generalized principle of self-support (FOGPSS) control strategy

Direct Synthesis of Iterative Algorithms With Bounds on Achievable Worst-Case Convergence Rate

Iterative first-order methods such as gradient descent and its variants are widely used for solving optimization and machine learning problems. There has been recent interest in analytic or numerically efficient methods for computing worst-case performance bounds for such algorithms, for example over the class of strongly convex loss functions. A popular approach is to assume the algorithm has a fixed size (fixed dimension, or memory) and that its structure is parameterized by one or two hyperparameters, for example a learning rate and a momentum parameter. Then, a Lyapunov function is sought to certify robust stability and subsequent optimization can be performed to find optimal hyperparameter tunings. In the present work, we instead fix the constraints that characterize the loss function and apply techniques from robust control synthesis to directly search over algorithms. This approach yields stronger results than those previously available, since the bounds produced hold over algorithms with an arbitrary, but finite, amount of memory rather than just holding for algorithms with a prescribed structure.Comment: American Control Conference, 202