A brief review of neural networks based learning and control and their applications for robots

As an imitation of the biological nervous systems, neural networks (NN), which are characterized with powerful learning ability, have been employed in a wide range of applications, such as control of complex nonlinear systems, optimization, system identification and patterns recognition etc. This article aims to bring a brief review of the state-of-art NN for the complex nonlinear systems. Recent progresses of NNs in both theoretical developments and practical applications are investigated and surveyed. Specifically, NN based robot learning and control applications were further reviewed, including NN based robot manipulator control, NN based human robot interaction and NN based behavior recognition and generation

Integrated feedback scheduling and control co-design for motion coordination of networked induction motor systems

This paper investigates the codesign of remote speed control and network scheduling for motion coordination of multiple induction motors through a shared communication network. An integrated feedback scheduling algorithm is designed to allocate the optimal sampling period and priority to each control loop to optimize the global performance of a networked control system (NCS), while satisfying the constraints of stability and schedulability. A speed synchronization method is incorporated into the scheduling algorithm to improve the speed synchronization performance of multiple induction motors. The rational gain of the network speed controllers is calculated using the Lyapunov theorem and tuned online by fuzzy logic to guarantee the robustness against complicated variations on the communication network. Furthermore, a state predictor is designed to compensate the time delay which occurred in data transmission from the sensor to the controller, as a part of the networked controller. Simulation results support the effectiveness of the proposed control-and-scheduling codesign approach

Model Predictive Control Based on Deep Learning for Solar Parabolic-Trough Plants

En la actualidad, cada vez es mayor el interés por utilizar energías renovables, entre las que se encuentra la energía solar. Las plantas de colectores cilindro-parabólicos son un tipo de planta termosolar donde se hace incidir la radiación del Sol en unos tubos mediante el uso de unos espejos con forma de parábola. En el interior de estos tubos circula un fluido, generalmente aceite o agua, que se calienta para generar vapor y hacer girar una turbina, produciendo energía eléctrica. Uno de los métodos más utilizados para manejar estas plantas es el control predictivo basado en modelo (model predictive control, MPC), cuyo funcionamiento consiste en obtener las señales de control óptimas que se enviarán a la planta basándose en el uso de un modelo de la misma. Este método permite predecir el estado que adoptará el sistema según la estrategia de control escogida a lo largo de un horizonte de tiempo. El MPC tiene como desventaja un gran coste computacional asociado a la resolución de un problema de optimización en cada instante de muestreo. Esto dificulta su implementación en plantas comerciales y de gran tamaño, por lo que, actualmente, uno de los principales retos es la disminución de estos tiempos de cálculo, ya sea tecnológicamente o mediante el uso de técnicas subóptimas que simplifiquen el problema. En este proyecto, se propone el uso de redes neuronales que aprendan offline de la salida proporcionada por un controlador predictivo para luego poder aproximarla. Se han entrenado diferentes redes neuronales utilizando un conjunto de datos de 30 días de simulación y modificando el número de entradas. Los resultados muestran que las redes neuronales son capaces de proporcionar prácticamente la misma potencia que el MPC con variaciones más suaves de la salida y muy bajas violaciones de las restricciones, incluso disminuyendo el número de entradas. El trabajo desarrollado se ha publicado en Renewable Energy, una revista del primer cuartil en Green & sustainable science & technology y Energy and fuels.Nowadays, there is an increasing interest in using renewable energy sources, including solar energy. Parabolic trough plants are a type of solar thermal power plant in which solar radiation is reflected onto tubes with parabolic mirrors. Inside these tubes circulates a fluid, usually oil or water, which is heated to generate steam and turn a turbine to produce electricity. One of the most widely used methods to control these plants is model predictive control (MPC), which obtains the optimal control signals to send to the plant based on the use of a model. This method makes it possible to predict its future state according to the chosen control strategy over a time horizon. The MPC has the disadvantage of a significant computational cost associated with resolving an optimization problem at each sampling time. This makes it challenging to implement in commercial and large plants, so currently, one of the main challenges is to reduce these computational times, either technologically or by using suboptimal techniques that simplify the problem. This project proposes the use of neural networks that learn offline from the output provided by a predictive controller to then approximate it. Different neural networks have been trained using a 30-day simulation dataset and modifying the number of irradiance and temperature inputs. The results show that the neural networks can provide practically the same power as the MPC with smoother variations of the output and very low violations of the constraints, even when decreasing the number of inputs. The work has been published in Renewable Energy, a first quartile journal in Green & sustainable science & technology and Energy and fuels.Universidad de Sevilla. Máster en Ingeniería Industria

Adaptive and learning-based formation control of swarm robots

Autonomous aerial and wheeled mobile robots play a major role in tasks such as search and rescue, transportation, monitoring, and inspection. However, these operations are faced with a few open challenges including robust autonomy, and adaptive coordination based on the environment and operating conditions, particularly in swarm robots with limited communication and perception capabilities. Furthermore, the computational complexity increases exponentially with the number of robots in the swarm. This thesis examines two different aspects of the formation control problem. On the one hand, we investigate how formation could be performed by swarm robots with limited communication and perception (e.g., Crazyflie nano quadrotor). On the other hand, we explore human-swarm interaction (HSI) and different shared-control mechanisms between human and swarm robots (e.g., BristleBot) for artistic creation. In particular, we combine bio-inspired (i.e., flocking, foraging) techniques with learning-based control strategies (using artificial neural networks) for adaptive control of multi- robots. We first review how learning-based control and networked dynamical systems can be used to assign distributed and decentralized policies to individual robots such that the desired formation emerges from their collective behavior. We proceed by presenting a novel flocking control for UAV swarm using deep reinforcement learning. We formulate the flocking formation problem as a partially observable Markov decision process (POMDP), and consider a leader-follower configuration, where consensus among all UAVs is used to train a shared control policy, and each UAV performs actions based on the local information it collects. In addition, to avoid collision among UAVs and guarantee flocking and navigation, a reward function is added with the global flocking maintenance, mutual reward, and a collision penalty. We adapt deep deterministic policy gradient (DDPG) with centralized training and decentralized execution to obtain the flocking control policy using actor-critic networks and a global state space matrix. In the context of swarm robotics in arts, we investigate how the formation paradigm can serve as an interaction modality for artists to aesthetically utilize swarms. In particular, we explore particle swarm optimization (PSO) and random walk to control the communication between a team of robots with swarming behavior for musical creation

Advances in Reinforcement Learning

Reinforcement Learning (RL) is a very dynamic area in terms of theory and application. This book brings together many different aspects of the current research on several fields associated to RL which has been growing rapidly, producing a wide variety of learning algorithms for different applications. Based on 24 Chapters, it covers a very broad variety of topics in RL and their application in autonomous systems. A set of chapters in this book provide a general overview of RL while other chapters focus mostly on the applications of RL paradigms: Game Theory, Multi-Agent Theory, Robotic, Networking Technologies, Vehicular Navigation, Medicine and Industrial Logistic

Formation control of mobile robots and unmanned aerial vehicles

In this dissertation, the nonlinear control of nonholonomic mobile robot formations and unmanned aerial vehicle (UAV) formations is undertaken and presented in six papers. In the first paper, an asymptotically stable combined kinematic/torque control law is developed for leader-follower based formation control of mobile robots using backstepping. A neural network (NN) is introduced along with robust integral of the sign of the error (RISE) feedback to approximate the dynamics of the follower as well as its leader using online weight tuning. Subsequently, in the second paper, a novel NN observer is designed to estimate the linear and angular velocities of both the follower and its leader robot and a NN output feedback control law is developed. On the other hand, in the third paper, a NN-based output feedback control law is presented for the control of an underactuated quad rotor UAV, and a NN virtual control input scheme is proposed which allows all six degrees of freedom to be controlled using only four control inputs. The results of this paper are extended to include the control of quadrotor UAV formations, and a novel three-dimensional leader-follower framework is proposed in the fourth paper. Next, in the fifth paper, the discrete-time nonlinear optimal control is undertaken using two online approximators (OLA\u27s) to solve the infinite horizon Hamilton-Jacobi-Bellman (HJB) equation forward-in-time to achieve nearly optimal regulation and tracking control. In contrast, paper six utilizes a single OLA to solve the infinite horizon HJB and Hamilton-Jacobi-Isaacs (HJI) equations forward-intime for the near optimal regulation and tracking control of continuous affine nonlinear systems. The effectiveness of the optimal tracking controllers proposed in the fifth and sixth papers are then demonstrated using nonholonomic mobile robot formation control --Abstract, page iv

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field

A Hybrid Nonlinear Model Predictive Control and Recurrent Neural Networks for Fault-Tolerant Control of an Autonomous Underwater Vehicle

The operation of Autonomous Unmanned Vehicles (AUVs) that is used for environment protection, risk evaluation and plan determination for emergency, are among the most important and challenging problems. An area that has received much attention for use of AUVs is in underwater applications where much work remains to be done to equip AUVs with systems to steer them accurately and reliably in harsh marine environments. Design of control strategies for AUVs is very challenging as compared to other systems due to their operational environment (ocean). Particularly when hydrodynamic parameters uncertainties are to be integrated into both the controller design as well as AUVs nonlinear dynamics. On the other hand, AUVs like all other mechanical systems are prone to faults. Dealing effectively with faulty situations for mechanical systems is an important consideration since faults can result in abnormal operation or even a failure. Hence, fault tolerant and fault-accommodating methods in the controller design are among active research topics for maintaining the reliability of complex AUV control systems. The objective of this thesis is to develop a nonlinear Model Predictive Control (MPC) that requires solving an online Quadratic Programming (QP) problem by using a Recurrent Neural Network (RNN). Also, an Extended Kalman Filter (EKF) is integrated with the developed scheme to provide the MPC algorithm with the system states estimates as well as a nonlinear prediction. This hybrid control approach utilizes both the mathematical model of the system as well as the adaptive nature of the intelligent technique through neural networks. The reason behind the selection of MPC is to benefit from its main capability in optimization within the current time slots while taking future time slots into consideration. The proposed control method is integrated with EKF which is an appropriate method for state estimation and data reconciliation of nonlinear systems. In order to address the high performance runtime cost of solving the MPC problem (formulated as a quadratic programming problem), an RNN is developed that has a low model complexity as well as good performance in real-time implementation. The proposed method is first developed to control an AUV following a desired trajectory. Since the problem of trajectory tracking and path following of AUVs exhibit nonlinear behavior, the effectiveness of the developed MPC-RNN algorithm is studied in comparison with two other control system methods, namely the linear MPC using Kalman Filter (KF) and the conventional nonlinear MPC using the EKF. In order to guarantee the fault-tolerant features of our proposed control method when faced with severe actuator faults, the developed MPC-RNN scheme is integrated with a dual Extended Kalman Filter that is used for a combined estimation of AUV states and parameters. The actuator faults are defined as the system parameters that are to be estimated online by the dual-EKF. Therefore, the developed Active Fault-Tolerant Control (AFTC) strategy is then applied to an AUV faced with loss of effectiveness (LOE) actuator fault scenarios while following a trajectory. Analysis and discussions regarding the comparison of the proposed AFTC method with Fault-Tolerant Nonlinear Model Predictive Control (FTNMPC) algorithm are presented in this work. The proposed approach to AFTC exploits the advantages of the MPC-RNN algorithm properties as well as accounting explicitly for severe control actuator faults in the nonlinear AUV model with uncertainties that are formulated by the MPC