Learning-Based Controller Design with Application to a Chiller Process

In this thesis, we present and study a few approaches for constructing controllers for uncertain systems, using a combination of classical control theory and modern machine learning methods. The thesis can be divided into two subtopics. The first, which is the focus of the first two papers, is dual control. The second, which is the focus of the third and last paper, is multiple-input multiple-output (MIMO) control of a chiller process. In dual control, the goal is to construct controllers for uncertain systems that in expectation minimize some cost over a certain time horizon. To achieve this, the controller must take into account the dual goals of accumulating more information about the process, by applying some probing input, and using the available information for controlling the system. This is referred to as the exploration-exploitation trade-off. Although optimal dual controllers in theory can be computed by solving a functional equation, this is usually intractable in practice, with only some simple special cases as exceptions. Therefore, it is interesting to examine methods for approximating optimal dual control. In the first paper, we take the approach of approximating the value function, which is the solution of the functional equation that can be used to deduce the optimal control, by using artificial neural networks. In the second paper, neural networks are used to represent and estimate hyperstates, which contain information about the conditional probability distributions of the system uncertainties. The optimal dual controller is a function of the hyperstate, and hence it should be useful to have a representation of this quantity when constructing an approximately optimal dual controller. The hyperstate transition model is used in combination with a reinforcement learning algorithm for constructing a dual controller from stochastic simulations of a system model that includes models of the system uncertainties. In the third paper, we suggest a simple reinforcement learning method that can be used to construct a decoupling matrix that allows MIMO control of a chiller process. Compared to the commonly used single-input single-output (SISO) structures, these controllers can decrease the variations in some system signals. This makes it possible to run the system at operating points closer to some constraints, which in turn can enable more energy-efficient operation

Optimal Control of Unknown Nonlinear System From Inputoutput Data

Optimal control designers usually require a plant model to design a controller. The problem is the controller\u27s performance heavily depends on the accuracy of the plant model. However, in many situations, it is very time-consuming to implement the system identification procedure and an accurate structure of a plant model is very difficult to obtain. On the other hand, neuro-fuzzy models with product inference engine, singleton fuzzifier, center average defuzzifier, and Gaussian membership functions can be easily trained by many well-established learning algorithms based on given input-output data pairs. Therefore, this kind of model is used in the current optimal controller design. Two approaches of designing optimal controllers of unknown nonlinear systems based on neuro-fuzzy models are presented in the thesis. The first approach first utilizes neuro-fuzzy models to approximate the unknown nonlinear systems, and then the feasible-direction algorithm is used to achieve the numerical solution of the Euler-Lagrange equations of the formulated optimal control problem. This algorithm uses the steepest descent to find the search direction and then apply a one-dimensional search routine to find the best step length. Finally several nonlinear optimal control problems are simulated and the results show that the performance of the proposed approach is quite similar to that of optimal control to the system represented by an explicit mathematical model. However, due to the limitation of the feasible-direction algorithm, this method cannot be applied to highly nonlinear and dimensional plants. Therefore, another approach that can overcome these drawbacks is proposed. This method utilizes Takagi-Sugeno (TS) fuzzy models to design the optimal controller. TS fuzzy models are first derived from the direct linearization of the neuro-fuzzy models, which is close to the local linearization of the nonlinear dynamic systems. The operating points are chosen so that the TS fuzzy model is a good approximation of the neuro-fuzzy model. Based on the TS fuzzy model, the optimal control is implemented for a nonlinear two-link flexible robot and a rigid asymmetric spacecraft, thus providing the possibility of implementing the well-established optimal control method on unknown nonlinear dynamic systems

Lyapunov based optimal control of a class of nonlinear systems

Optimal control of nonlinear systems is in fact difficult since it requires the solution to the Hamilton-Jacobi-Bellman (HJB) equation which has no closed-form solution. In contrast to offline and/or online iterative schemes for optimal control, this dissertation in the form of five papers focuses on the design of iteration free, online optimal adaptive controllers for nonlinear discrete and continuous-time systems whose dynamics are completely or partially unknown even when the states not measurable. Thus, in Paper I, motivated by homogeneous charge compression ignition (HCCI) engine dynamics, a neural network-based infinite horizon robust optimal controller is introduced for uncertain nonaffine nonlinear discrete-time systems. First, the nonaffine system is transformed into an affine-like representation while the resulting higher order terms are mitigated by using a robust term. The optimal adaptive controller for the affinelike system solves HJB equation and identifies the system dynamics provided a target set point is given. Since it is difficult to define the set point a priori in Paper II, an extremum seeking control loop is designed while maximizing an uncertain output function. On the other hand, Paper III focuses on the infinite horizon online optimal tracking control of known nonlinear continuous-time systems in strict feedback form by using state and output feedback by relaxing the initial admissible controller requirement. Paper IV applies the optimal controller from Paper III to an underactuated helicopter attitude and position tracking problem. In Paper V, the optimal control of nonlinear continuous-time systems in strict feedback form from Paper III is revisited by using state and output feedback when the internal dynamics are unknown. Closed-loop stability is demonstrated for all the controller designs developed in this dissertation by using Lyapunov analysis --Abstract, page iv

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Knowledge Transfer Between Robots with Similar Dynamics for High-Accuracy Impromptu Trajectory Tracking

In this paper, we propose an online learning approach that enables the inverse dynamics model learned for a source robot to be transferred to a target robot (e.g., from one quadrotor to another quadrotor with different mass or aerodynamic properties). The goal is to leverage knowledge from the source robot such that the target robot achieves high-accuracy trajectory tracking on arbitrary trajectories from the first attempt with minimal data recollection and training. Most existing approaches for multi-robot knowledge transfer are based on post-analysis of datasets collected from both robots. In this work, we study the feasibility of impromptu transfer of models across robots by learning an error prediction module online. In particular, we analytically derive the form of the mapping to be learned by the online module for exact tracking, propose an approach for characterizing similarity between robots, and use these results to analyze the stability of the overall system. The proposed approach is illustrated in simulation and verified experimentally on two different quadrotors performing impromptu trajectory tracking tasks, where the quadrotors are required to accurately track arbitrary hand-drawn trajectories from the first attempt.Comment: European Control Conference (ECC) 201

Adaptation and Learning for Manipulators and Machining

This thesis presents methods for improving the accuracy and efficiency of tasks performed using different kinds of industrial manipulators, with a focus on the application of machining. Industrial robots offer a flexible and cost-efficient alternative to machine tools for machining, but cannot achieve as high accuracy out of the box. This is mainly caused by non-ideal properties in the robot joints such as backlash and compliance, in combination with the strong process forces that affect the robot during machining operations. In this thesis, three different approaches to improving the robotic machining accuracy are presented. First, a macro/micro-manipulator approach is considered, where an external compensation mechanism is used in combination with the robot, for compensation of high-frequency Cartesian errors. Two different milling scenarios are evaluated, where a significant increase in accuracy was obtained. The accuracy specification of 50 μm was reached for both scenarios. Because of the limited workspace and the higher bandwidth of the compensation mechanism compared to the robot, two different mid-ranging approaches for control of the relative position between the robot and the compensator are developed and evaluated. Second, modeling and identification of robot joints is considered. The proposed method relies on clamping the manipulator end effector and actuating the joints, while measuring joint motor torque and motor position. The joint stiffness and backlash can subsequently be extracted from the measurements, to be used for compensation of the deflections that occur during machining. Third, a model-based iterative learning control (ILC) approach is proposed, where feedback is provided from three different sensors of varying investment costs. Using position measurements from an optical tracking system, an error decrease of up to 84 % was obtained. Measurements of end-effector forces yielded an error decrease of 55 %, and a force-estimation method based on joint motor torques decreased the error by 38 %. Further investigation of ILC methods is considered for a different kind of manipulator, a marine vibrator, for the application of marine seismic acquisition. A frequency-domain ILC strategy is proposed, in order to attenuate undesired overtones and improve the tracking accuracy. The harmonics were suppressed after approximately 20 iterations of the ILC algorithm, and the absolute tracking error was r educed by a factor of approximately 50. The final problem considered in this thesis concerns increasing the efficiency of machining tasks, by minimizing cycle times. A force-control approach is proposed to maximize the feed rate, and a learning algorithm for path planning of the machining path is employed for the case of machining in non-isotropic materials, such as wood. The cycle time was decreased by 14 % with the use of force control, and on average an additional 28 % decrease was achieved by use of a learning algorithm. Furthermore, by means of reinforcement learning, the path-planning algorithm is refined to provide optimal solutions and to incorporate an increased number of machining directions