Theory and Applications of Robust Optimization

In this paper we survey the primary research, both theoretical and applied, in the area of Robust Optimization (RO). Our focus is on the computational attractiveness of RO approaches, as well as the modeling power and broad applicability of the methodology. In addition to surveying prominent theoretical results of RO, we also present some recent results linking RO to adaptable models for multi-stage decision-making problems. Finally, we highlight applications of RO across a wide spectrum of domains, including finance, statistics, learning, and various areas of engineering.Comment: 50 page

Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control

Trial-and-error based reinforcement learning (RL) has seen rapid advancements in recent times, especially with the advent of deep neural networks. However, the majority of autonomous RL algorithms require a large number of interactions with the environment. A large number of interactions may be impractical in many real-world applications, such as robotics, and many practical systems have to obey limitations in the form of state space or control constraints. To reduce the number of system interactions while simultaneously handling constraints, we propose a model-based RL framework based on probabilistic Model Predictive Control (MPC). In particular, we propose to learn a probabilistic transition model using Gaussian Processes (GPs) to incorporate model uncertainty into long-term predictions, thereby, reducing the impact of model errors. We then use MPC to find a control sequence that minimises the expected long-term cost. We provide theoretical guarantees for first-order optimality in the GP-based transition models with deterministic approximate inference for long-term planning. We demonstrate that our approach does not only achieve state-of-the-art data efficiency, but also is a principled way for RL in constrained environments.Comment: Accepted at AISTATS 2018

Random Finite Set Theory and Optimal Control of Large Collaborative Swarms

Controlling large swarms of robotic agents has many challenges including, but not limited to, computational complexity due to the number of agents, uncertainty in the functionality of each agent in the swarm, and uncertainty in the swarm's configuration. This work generalizes the swarm state using Random Finite Set (RFS) theory and solves the control problem using Model Predictive Control (MPC) to overcome the aforementioned challenges. Computationally efficient solutions are obtained via the Iterative Linear Quadratic Regulator (ILQR). Information divergence is used to define the distance between the swarm RFS and the desired swarm configuration. Then, a stochastic optimal control problem is formulated using a modified L2^2 distance. Simulation results using MPC and ILQR show that swarm intensities converge to a target destination, and the RFS control formulation can vary in the number of target destinations. ILQR also provides a more computationally efficient solution to the RFS swarm problem when compared to the MPC solution. Lastly, the RFS control solution is applied to a spacecraft relative motion problem showing the viability for this real-world scenario.Comment: arXiv admin note: text overlap with arXiv:1801.0731

Stochastic optimal adaptive controller and communication protocol design for networked control systems

Networked Control System (NCS) is a recent topic of research wherein the feedback control loops are closed through a real-time communication network. Many design challenges surface in such systems due to network imperfections such as random delays, packet losses, quantization effects and so on. Since existing control techniques are unsuitable for such systems, in this dissertation, a suite of novel stochastic optimal adaptive design methodologies is undertaken for both linear and nonlinear NCS in presence of uncertain system dynamics and unknown network imperfections such as network-induced delays and packet losses. The design is introduced in five papers. In Paper 1, a stochastic optimal adaptive control design is developed for unknown linear NCS with uncertain system dynamics and unknown network imperfections. A value function is adjusted forward-in-time and online, and a novel update law is proposed for tuning value function estimator parameters. Additionally, by using estimated value function, optimal adaptive control law is derived based on adaptive dynamic programming technique. Subsequently, this design methodology is extended to solve stochastic optimal strategies of linear NCS zero-sum games in Paper 2. Since most systems are inherently nonlinear, a novel stochastic optimal adaptive control scheme is then developed in Paper 3 for nonlinear NCS with unknown network imperfections. On the other hand, in Paper 4, the network protocol behavior (e.g. TCP and UDP) are considered and optimal adaptive control design is revisited using output feedback for linear NCS. Finally, Paper 5 explores a co-design framework where both the controller and network scheduling protocol designs are addressed jointly so that proposed scheme can be implemented into next generation Cyber Physical Systems --Abstract, page iv