14,360 research outputs found
Reinforcement Learning for UAV Attitude Control
Autopilot systems are typically composed of an "inner loop" providing
stability and control, while an "outer loop" is responsible for mission-level
objectives, e.g. way-point navigation. Autopilot systems for UAVs are
predominately implemented using Proportional, Integral Derivative (PID) control
systems, which have demonstrated exceptional performance in stable
environments. However more sophisticated control is required to operate in
unpredictable, and harsh environments. Intelligent flight control systems is an
active area of research addressing limitations of PID control most recently
through the use of reinforcement learning (RL) which has had success in other
applications such as robotics. However previous work has focused primarily on
using RL at the mission-level controller. In this work, we investigate the
performance and accuracy of the inner control loop providing attitude control
when using intelligent flight control systems trained with the state-of-the-art
RL algorithms, Deep Deterministic Gradient Policy (DDGP), Trust Region Policy
Optimization (TRPO) and Proximal Policy Optimization (PPO). To investigate
these unknowns we first developed an open-source high-fidelity simulation
environment to train a flight controller attitude control of a quadrotor
through RL. We then use our environment to compare their performance to that of
a PID controller to identify if using RL is appropriate in high-precision,
time-critical flight control.Comment: 13 pages, 9 figure
Neural-Network Vector Controller for Permanent-Magnet Synchronous Motor Drives: Simulated and Hardware-Validated Results
This paper focuses on current control in a permanentmagnet synchronous motor (PMSM). The paper has two main objectives: The first objective is to develop a neural-network (NN) vector controller to overcome the decoupling inaccuracy problem associated with conventional PI-based vector-control methods. The NN is developed using the full dynamic equation of a PMSM, and trained to implement optimal control based on approximate dynamic programming. The second objective is to evaluate the robust and adaptive performance of the NN controller against that of the conventional standard vector controller under motor parameter variation and dynamic control conditions by (a) simulating the behavior of a PMSM typically used in realistic electric vehicle applications and (b) building an experimental system for hardware validation as well as combined hardware and simulation evaluation. The results demonstrate that the NN controller outperforms conventional vector controllers in both simulation and hardware implementation
Feedback control by online learning an inverse model
A model, predictor, or error estimator is often used by a feedback controller to control a plant. Creating such a model is difficult when the plant exhibits nonlinear behavior. In this paper, a novel online learning control framework is proposed that does not require explicit knowledge about the plant. This framework uses two learning modules, one for creating an inverse model, and the other for actually controlling the plant. Except for their inputs, they are identical. The inverse model learns by the exploration performed by the not yet fully trained controller, while the actual controller is based on the currently learned model. The proposed framework allows fast online learning of an accurate controller. The controller can be applied on a broad range of tasks with different dynamic characteristics. We validate this claim by applying our control framework on several control tasks: 1) the heating tank problem (slow nonlinear dynamics); 2) flight pitch control (slow linear dynamics); and 3) the balancing problem of a double inverted pendulum (fast linear and nonlinear dynamics). The results of these experiments show that fast learning and accurate control can be achieved. Furthermore, a comparison is made with some classical control approaches, and observations concerning convergence and stability are made
Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures
Reinforcement learning (RL) constitutes a promising solution for alleviating
the problem of traffic congestion. In particular, deep RL algorithms have been
shown to produce adaptive traffic signal controllers that outperform
conventional systems. However, in order to be reliable in highly dynamic urban
areas, such controllers need to be robust with the respect to a series of
exogenous sources of uncertainty. In this paper, we develop an open-source
callback-based framework for promoting the flexible evaluation of different
deep RL configurations under a traffic simulation environment. With this
framework, we investigate how deep RL-based adaptive traffic controllers
perform under different scenarios, namely under demand surges caused by special
events, capacity reductions from incidents and sensor failures. We extract
several key insights for the development of robust deep RL algorithms for
traffic control and propose concrete designs to mitigate the impact of the
considered exogenous uncertainties.Comment: 8 page
Comparative evaluation of approaches in T.4.1-4.3 and working definition of adaptive module
The goal of this deliverable is two-fold: (1) to present and compare different approaches towards learning and encoding movements us- ing dynamical systems that have been developed by the AMARSi partners (in the past during the first 6 months of the project), and (2) to analyze their suitability to be used as adaptive modules, i.e. as building blocks for the complete architecture that will be devel- oped in the project. The document presents a total of eight approaches, in two groups: modules for discrete movements (i.e. with a clear goal where the movement stops) and for rhythmic movements (i.e. which exhibit periodicity). The basic formulation of each approach is presented together with some illustrative simulation results. Key character- istics such as the type of dynamical behavior, learning algorithm, generalization properties, stability analysis are then discussed for each approach. We then make a comparative analysis of the different approaches by comparing these characteristics and discussing their suitability for the AMARSi project
- …