Genetic algorithm optimization and control system design of flexible structures

This paper presents an investigation into the deployment of genetic algorithm (GA)-based controller design and optimization for vibration suppression in flexible structures. The potential of GA is explored in three case studies. In the first case study, the potential of GA is demonstrated in the development and optimization of a hybrid learning control scheme for vibration control of flexible manipulators. In the second case study, an active control mechanism for vibration suppression of flexible beam structures using GA optimization technique is proposed. The third case study presents the development of an effective adaptive command shaping control scheme for vibration control of a twin rotor system, where GA is employed to optimize the amplitudes and time locations of the impulses in the proposed control algorithm. The effectiveness of the proposed control schemes is verified in both an experimental and a simulation environment, and their performances are assessed in both the time and frequency domains

Classifying intelligence in machines : a taxonomy of intelligent control

The quest to create machines that can solve problems as humans do leads us to intelligent control. This field encompasses control systems that can adapt to changes and learn to improve their actions—traits typically associated with human intelligence. In this work we seek to determine how intelligent these classes of control systems are by quantifying their level of adaptability and learning. First we describe the stages of development towards intelligent control and present a definition based on literature. Based on the key elements of this definition, we propose a novel taxonomy of intelligent control methods, which assesses the extent to which they handle uncertainties in three areas: the environment, the controller, and the goals. This taxonomy is applicable to a variety of robotic and other autonomous systems, which we demonstrate through several examples of intelligent control methods and their classifications. Looking at the spread of classifications based on this taxonomy can help researchers identify where control systems can be made more intelligent

Improving the efficiency of reinforcement learning for a spacecraft powered descent with Q-learning

Reinforcement learning entails many intuitive and useful approaches to solving various problems. Its main premise is to learn how to complete tasks by interacting with the environment and observing which actions are more optimal with respect to a reward signal. Methods from reinforcement learning have long been applied in aerospace and have more recently seen renewed interest in space applications. Problems in spacecraft control can benefit from the use of intelligent techniques when faced with significant uncertainties - as is common for space environments. Solving these control problems using reinforcement learning remains a challenge partly due to long training times and sensitivity in performance to hyperparameters which require careful tuning. In this work we seek to address both issues for a sample spacecraft control problem. To reduce training times compared to other approaches, we simplify the problem by discretising the action space and use a data-efficient algorithm to train the agent. Furthermore, we employ an automated approach to hyperparameter selection which optimises for a specified performance metric. Our approach is tested on a 3-DOF powered descent problem with uncertainties in the initial conditions. We run experiments with two different problem formulations - using a 'shaped' state representation to guide the agent and also a 'raw' state representation with unprocessed values of position, velocity and mass. The results show that an agent can learn a near-optimal policy efficiently by appropriately defining the action-space and state-space. Using the raw state representation led to 'reward-hacking' and poor performance, which highlights the importance of the problem and state-space formulation in successfully training reinforcement learning agents. In addition, we show that the optimal hyperparameters can vary significantly based on the choice of loss function. Using two sets of hyperparameters optimised for different loss functions, we demonstrate that in both cases the agent can find near-optimal policies with comparable performance to previously applied methods

Model Predictive Control Based on Deep Learning for Solar Parabolic-Trough Plants

En la actualidad, cada vez es mayor el interés por utilizar energías renovables, entre las que se encuentra la energía solar. Las plantas de colectores cilindro-parabólicos son un tipo de planta termosolar donde se hace incidir la radiación del Sol en unos tubos mediante el uso de unos espejos con forma de parábola. En el interior de estos tubos circula un fluido, generalmente aceite o agua, que se calienta para generar vapor y hacer girar una turbina, produciendo energía eléctrica. Uno de los métodos más utilizados para manejar estas plantas es el control predictivo basado en modelo (model predictive control, MPC), cuyo funcionamiento consiste en obtener las señales de control óptimas que se enviarán a la planta basándose en el uso de un modelo de la misma. Este método permite predecir el estado que adoptará el sistema según la estrategia de control escogida a lo largo de un horizonte de tiempo. El MPC tiene como desventaja un gran coste computacional asociado a la resolución de un problema de optimización en cada instante de muestreo. Esto dificulta su implementación en plantas comerciales y de gran tamaño, por lo que, actualmente, uno de los principales retos es la disminución de estos tiempos de cálculo, ya sea tecnológicamente o mediante el uso de técnicas subóptimas que simplifiquen el problema. En este proyecto, se propone el uso de redes neuronales que aprendan offline de la salida proporcionada por un controlador predictivo para luego poder aproximarla. Se han entrenado diferentes redes neuronales utilizando un conjunto de datos de 30 días de simulación y modificando el número de entradas. Los resultados muestran que las redes neuronales son capaces de proporcionar prácticamente la misma potencia que el MPC con variaciones más suaves de la salida y muy bajas violaciones de las restricciones, incluso disminuyendo el número de entradas. El trabajo desarrollado se ha publicado en Renewable Energy, una revista del primer cuartil en Green & sustainable science & technology y Energy and fuels.Nowadays, there is an increasing interest in using renewable energy sources, including solar energy. Parabolic trough plants are a type of solar thermal power plant in which solar radiation is reflected onto tubes with parabolic mirrors. Inside these tubes circulates a fluid, usually oil or water, which is heated to generate steam and turn a turbine to produce electricity. One of the most widely used methods to control these plants is model predictive control (MPC), which obtains the optimal control signals to send to the plant based on the use of a model. This method makes it possible to predict its future state according to the chosen control strategy over a time horizon. The MPC has the disadvantage of a significant computational cost associated with resolving an optimization problem at each sampling time. This makes it challenging to implement in commercial and large plants, so currently, one of the main challenges is to reduce these computational times, either technologically or by using suboptimal techniques that simplify the problem. This project proposes the use of neural networks that learn offline from the output provided by a predictive controller to then approximate it. Different neural networks have been trained using a 30-day simulation dataset and modifying the number of irradiance and temperature inputs. The results show that the neural networks can provide practically the same power as the MPC with smoother variations of the output and very low violations of the constraints, even when decreasing the number of inputs. The work has been published in Renewable Energy, a first quartile journal in Green & sustainable science & technology and Energy and fuels.Universidad de Sevilla. Máster en Ingeniería Industria

Neural networks in control engineering

The purpose of this thesis is to investigate the viability of integrating neural networks into control structures. These networks are an attempt to create artificial intelligent systems with the ability to learn and remember. They mathematically model the biological structure of the brain and consist of a large number of simple interconnected processing units emulating brain cells. Due to the highly parallel and consequently computationally expensive nature of these networks, intensive research in this field has only become feasible due to the availability of powerful personal computers in recent years. Consequently, attempts at exploiting the attractive learning and nonlinear optimization characteristics of neural networks have been made in most fields of science and engineering, including process control. The control structures suggested in the literature for the inclusion of neural networks in control applications can be divided into four major classes. The first class includes approaches in which the network forms part of an adaptive mechanism which modulates the structure or parameters of the controller. In the second class the network forms part of the control loop and replaces the conventional control block, thus leading to a pure neural network control law. The third class consists of topologies in which neural networks are used to produce models of the system which are then utilized in the control structure, whilst the fourth category includes suggestions which are specific to the problem or system structure and not suitable for a generic neural network-based-approach to control problems. Although several of these approaches show promising results, only model based structures are evaluated in this thesis. This is due to the fact that many of the topologies in other classes require system estimation to produce the desired network output during training, whereas the training data for network models is obtained directly by sampling the system input(s) and output(s). Furthermore, many suggested structures lack the mathematical motivation to consider them for a general structure, whilst the neural network model topologies form natural extensions of their linear model based origins. Since it is impractical and often impossible to collect sufficient training data prior to implementing the neural network based control structure, the network models have to be suited to on-line training during operation. This limits the choice of network topologies for models to those that can be trained on a sample by sample basis (pattern learning) and furthermore are capable of learning even when the variation in training data is relatively slow as is the case for most controlled dynamic systems. A study of feedforward topologies (one of the main classes of networks) shows that the multilayer perceptron network with its backpropagation training is well suited to model nonlinear mappings but fails to learn and generalize when subjected to slow varying training data. This is due to the global input interpretation of this structure, in which any input affects all hidden nodes such that no effective partitioning of the input space can be achieved. This problem is overcome in a less flexible feedforward structure, known as regular Gaussian network. In this network, the response of each hidden node is limited to a -sphere around its center and these centers are fixed in a uniform distribution over the entire input space. Each input to such a network is therefore interpreted locally and only effects nodes with their centers in close proximity. A deficiency common to all feedforward networks, when considered as models for dynamic systems, is their inability to conserve previous outputs and states for future predictions. Since this absence of dynamic capability requires the user to identify the order of the system prior to training and is therefore not entirely self-learning, more advanced network topologies are investigated. The most versatile of these structures, known as a fully recurrent network, re-uses the previous state of each of its nodes for subsequent outputs. However, despite its superior modelling capability, the tests performed using the Williams and Zipser training algorithm show that such structures often fail to converge and require excessive computing power and time, when increased in size. Despite its rigid structure and lack of dynamic capability, the regular Gaussian network produces the most reliable and robust models and was therefore selected for the evaluations in this study. To overcome the network initialization problem, found when using a pure neural network model, a combination structure· _in which the network operates in parallel with a mathematical model is suggested. This approach allows the controller to be implemented without any prior network training and initially relies purely on the mathematical model, much like conventional approaches. The network portion is then trained during on-line operation in order to improve the model. Once trained, the enhanced model can be used to improve the system response, since model exactness plays an important role in the control action achievable with model based structures. The applicability of control structures based on neural network models is evaluated by comparing the performance of two network approaches to that of a linear structure, using a simulation of a nonlinear tank system. The first network controller is developed from the internal model control (IMC) structure, which includes a forward and inverse model of the system to be controlled. Both models can be replaced by a combination of mathematical and neural topologies, the network portion of which is trained on-line to compensate for the discrepancies between the linear model _ and nonlinear system. Since the network has no dynamic ·capacity, .former system outputs are used as inputs to the forward and inverse model. Due to this direct feedback, the trained structure can be tuned to perform within limits not achievable using a conventional linear system. As mentioned previously the IMC structure uses both forward and inverse models. Since the control law requires that these models are exact inverses, an iterative inversion algorithm has to be used to improve the values produced by the inverse combination model. Due to deadtimes and right-half-plane zeroes, many systems are furthermore not directly invertible. Whilst such unstable elements can be removed from mathematical models, the inverse network is trained directly from the forward model and can not be compensated. These problems could be overcome by a control structure for which only a forward model is required. The neural predictive controller (NPC) presents such a topology. Based on the optimal control philosophy, this structure uses a model to predict several future outputs. The errors between these and the desired output are then collected to form the cost function, which may also include other factors such as the magnitude of the change in input. The input value that optimally fulfils all the objectives used to formulate the cost function, can then be found by locating its minimum. Since the model in this structure includes a neural network, the optimization can not be formulated in a closed mathematical form and has to be performed using a numerical method. For the NPC topology, as for the neural network IMC structure, former system outputs are fed back to the model and again the trained network approach produces results not achievable with a linear model. Due to the single network approach, the NPC topology furthermore overcomes the limitations described for the neural network IMC structure and can be extended to include multivariable systems. This study shows that the nonlinear modelling capability of neural networks can be exploited to produce learning control structures with improved responses for nonlinear systems. Many of the difficulties described are due to the computational burden of these networks and associated algorithms. These are likely to become less significant due to the rapid development in computer technology and advances in neural network hardware. Although neural network based control structures are unlikely to replace the well understood linear topologies, which are adequate for the majority of applications, they might present a practical alternative where (due to nonlinearity or modelling errors) the conventional controller can not achieve the required control action

Online Learning of a Neural Fuel Control System for Gaseous Fueled SI Engines

This dissertation presents a new type of fuel control algorithm for gaseous fuelled vehicles. Gaseous fuels such as hydrogen and natural gas have been shown to be less polluting than liquid fuels such as gasoline, both at the tailpipe and on a total cycle basis. Unfortunately, it can be expensive to convert vehicles to gaseous fuels, partially due to small production runs for these vehicles. One of major development costs for a new vehicle is the development and calibration of the fuel controller. The research presented here includes a fuel controller which does not require an expensive calibration phase.The controller is based upon a two-part model, separating steady state and dynamic effects. This model is then used to estimate the optimum fuelling for the measured operating condition. The steady state model is calculated using an artificial neural network with an online learning scheme, allowing the model to continually update to improve the controller's performance. This is important during both the initial learning of the characteristics of a new engine, as well as tracking changes due to wear or damage.The dynamic model of the system is concerned with the significant transport delay between the time the fuel is injected and when the exhaust gas oxygen sensor makes the reading. One significant result of this research is the realization that a previous commonly used model for this delay has become significantly less accurate due to the shift from carburettors or central point injection to port injection.In addition to a description of the control scheme used, this dissertation includes a new method of algebraically inverting a neural network, avoiding computationally expensive iterative methods of optimizing the model. This can greatly speed up the control loop (or allow for less expensive, slower hardware).An important feature of a fuel control scheme is that it produces a small, stable limit cycle between rich and lean fuel-air mixtures. This dissertation expands the currently available models for the limit cycle characteristics of a system with a linear controller as well as developing a similar model for the neural network controller by linearizing the learning scheme.One of the most important aspects of this research is an experimental test, in which the controller was installed on a truck fuelled by natural gas. The tailpipe emissions of the truck with the new controller showed better results than the OEM controller on both carbon monoxide and nitrogen oxides, and the controller required no calibration and very little information about the properties of the engine.The significant original contributions resulting from this research include: -collection and summarization of previous work, -development of a method of automatically determining the pure time delay between the fuel injection event and the feedback measurement, -development of a more accurate model for the variability of the transport delay in modern port injection engines, -developing a fuel-air controller requiring minimal knowledge of the engine's parameters, -development of a method of algebraically inverting a neural network which is much faster than previous iterative methods, -demonstrating how to initialize the neural model by taking advantage of some important characteristics of the system, -expansion of the models available for the limit cycle produced by a system with a binary sensor and delay to include integral controllers with asymmetrical gains, -development of a limit cycle model for the new neural controller, and -experimental verification of the controller's tailpipe emissions performance, which compares favourably to the OEM controller

The application of neural networks to spacecraft control

This thesis investigates how two neural network-based control techniques can be applied to a specific spacecraft control problem. The neural networks used are simple backpropagation networks, consisting of one or more tansigmoidal neurons (neurons with tanh transfer functions) in a hidden layer, and a linear neuron in the output layer. The neural network control techniques investigated here are Direct Model Inversion and Indirect Model Inversion. The spacecraft control problem is that of reducing the vibrations of a spacecraft payload. The source of the vibrations is a mass imbalance in one of the reaction wheels of the spacecraft. Four components are represented in the spacecraft model. These are rigid body inertia, solar array flexure, fuel slosh and payload vibration. A simple sinusoidal signal is used to model the disturbance torque produced by the reaction wheel mass imbalance. The complete model is broadly based on the Solar Heliospheric Observatory (SOHO) that is due for launch in 1995. Each of the neural network control techniques used is shown to be successful in reducing the effects of the disturbance torques on the spacecraft payload. However, in each case, a simple positional feedback gain term provides more effective and reliable control.MPhi