Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control

Trial-and-error based reinforcement learning (RL) has seen rapid advancements in recent times, especially with the advent of deep neural networks. However, the majority of autonomous RL algorithms require a large number of interactions with the environment. A large number of interactions may be impractical in many real-world applications, such as robotics, and many practical systems have to obey limitations in the form of state space or control constraints. To reduce the number of system interactions while simultaneously handling constraints, we propose a model-based RL framework based on probabilistic Model Predictive Control (MPC). In particular, we propose to learn a probabilistic transition model using Gaussian Processes (GPs) to incorporate model uncertainty into long-term predictions, thereby, reducing the impact of model errors. We then use MPC to find a control sequence that minimises the expected long-term cost. We provide theoretical guarantees for first-order optimality in the GP-based transition models with deterministic approximate inference for long-term planning. We demonstrate that our approach does not only achieve state-of-the-art data efficiency, but also is a principled way for RL in constrained environments.Comment: Accepted at AISTATS 2018

Differentiable Robust Model Predictive Control

Deterministic model predictive control (MPC), while powerful, is often insufficient for effectively controlling autonomous systems in the real-world. Factors such as environmental noise and model error can cause deviations from the expected nominal performance. Robust MPC algorithms aim to bridge this gap between deterministic and uncertain control. However, these methods are often excessively difficult to tune for robustness due to the nonlinear and non-intuitive effects that controller parameters have on performance. To address this challenge, a unifying perspective on differentiable optimization for control is presented, which enables derivation of a general, differentiable tube-based MPC algorithm. The proposed approach facilitates the automatic and real-time tuning of robust controllers in the presence of large uncertainties and disturbances

Trajectory optimization and motion planning for quadrotors in unstructured environments

Trajectory optimization and motion planning for quadrotors in unstructured environments Coming out from university labs robots perform tasks usually navigating through unstructured environment. The realization of autonomous motion in such type of environments poses a number of challenges compared to highly controlled laboratory spaces. In unstructured environments robots cannot rely on complete knowledge of their sorroundings and they have to continously acquire information for decision making. The challenges presented are a consequence of the high-dimensionality of the state-space and of the uncertainty introduced by modeling and perception. This is even more true for aerial-robots that has a complex nonlinear dynamics a can move freely in 3D-space. To avoid this complexity a robot have to select a small set of relevant features, reason on a reduced state space and plan trajectories on short-time horizon. This thesis is a contribution towards the autonomous navigation of aerial robots (quadrotors) in real-world unstructured scenarios. The first three chapters present a contribution towards an implementation of Receding Time Horizon Optimal Control. The optimization problem for a model based trajectory generation in environments with obstacles is set, using an approach based on variational calculus and modeling the robots in the SE(3) Lie Group of 3D space transformations. The fourth chapter explores the problem of using minimal information and sensing to generate motion towards a goal in an indoor bulding-like scenario. The fifth chapter investigate the problem of extracting visual features from the environment to control the motion in an indoor corridor-like scenario. The last chapter deals with the problem of spatial reasoning and motion planning using atomic proposition in a multi-robot environments with obstacles

STABILITY AND PERFORMANCE OF NETWORKED CONTROL SYSTEMS

Network control systems (NCSs), as one of the most active research areas, are arousing comprehensive concerns along with the rapid development of network. This dissertation mainly discusses the stability and performance of NCSs into the following two parts. In the first part, a new approach is proposed to reduce the data transmitted in networked control systems (NCSs) via model reduction method. Up to our best knowledge, we are the first to propose this new approach in the scientific and engineering society. The "unimportant" information of system states vector is truncated by balanced truncation method (BTM) before sending to the networked controller via network based on the balance property of the remote controlled plant controllability and observability. Then, the exponential stability condition of the truncated NCSs is derived via linear matrix inequality (LMI) forms. This method of data truncation can usually reduce the time delay and further improve the performance of the NCSs. In addition, all the above results are extended to the switched NCSs. The second part presents a new robust sliding mode control (SMC) method for general uncertain time-varying delay stochastic systems with structural uncertainties and the Brownian noise (Wiener process). The key features of the proposed method are to apply singular value decomposition (SVD) to all structural uncertainties, to introduce adjustable parameters for control design along with the SMC method, and new Lyapunov-type functional. Then, a less-conservative condition for robust stability and a new robust controller for the general uncertain stochastic systems are derived via linear matrix inequality (LMI) forms. The system states are able to reach the SMC switching surface as guaranteed in probability 1 by the proposed control rule. Furthermore, the novel Lyapunov-type functional for the uncertain stochastic systems is used to design a new robust control for the general case where the derivative of time-varying delay can be any bounded value (e.g., greater than one). It is theoretically proved that the conservatism of the proposed method is less than the previous methods. All theoretical proofs are presented in the dissertation. The simulations validate the correctness of the theoretical results and have better performance than the existing results

A Neural Network Approach for Real-Time High-Dimensional Optimal Control

We propose a neural network approach for solving high-dimensional optimal control problems arising in real-time applications. Our approach yields controls in a feedback form, where the policy function is given by a neural network (NN). Specifically, we fuse the Hamilton-Jacobi-Bellman (HJB) and Pontryagin Maximum Principle (PMP) approaches by parameterizing the value function with an NN. We can therefore synthesize controls in real-time without having to solve an optimization problem. Once the policy function is trained, generating a control at a given space-time location takes milliseconds; in contrast, efficient nonlinear programming methods typically perform the same task in seconds. We train the NN offline using the objective function of the control problem and penalty terms that enforce the HJB equations. Therefore, our training algorithm does not involve data generated by another algorithm. By training on a distribution of initial states, we ensure the controls' optimality on a large portion of the state-space. Our grid-free approach scales efficiently to dimensions where grids become impractical or infeasible. We demonstrate the effectiveness of our approach on several multi-agent collision-avoidance problems in up to 150 dimensions. Furthermore, we empirically observe that the number of parameters in our approach scales linearly with the dimension of the control problem, thereby mitigating the curse of dimensionality.Comment: 16 pages, 12 figures. This work has been submitted for possible publication. Copyright may be transferred without notice, after which this version may no longer be availabl