413 research outputs found
Reinforcement Learning for UAV Attitude Control
Autopilot systems are typically composed of an "inner loop" providing
stability and control, while an "outer loop" is responsible for mission-level
objectives, e.g. way-point navigation. Autopilot systems for UAVs are
predominately implemented using Proportional, Integral Derivative (PID) control
systems, which have demonstrated exceptional performance in stable
environments. However more sophisticated control is required to operate in
unpredictable, and harsh environments. Intelligent flight control systems is an
active area of research addressing limitations of PID control most recently
through the use of reinforcement learning (RL) which has had success in other
applications such as robotics. However previous work has focused primarily on
using RL at the mission-level controller. In this work, we investigate the
performance and accuracy of the inner control loop providing attitude control
when using intelligent flight control systems trained with the state-of-the-art
RL algorithms, Deep Deterministic Gradient Policy (DDGP), Trust Region Policy
Optimization (TRPO) and Proximal Policy Optimization (PPO). To investigate
these unknowns we first developed an open-source high-fidelity simulation
environment to train a flight controller attitude control of a quadrotor
through RL. We then use our environment to compare their performance to that of
a PID controller to identify if using RL is appropriate in high-precision,
time-critical flight control.Comment: 13 pages, 9 figure
Model Predictive Control for Micro Aerial Vehicles: A Survey
This paper presents a review of the design and application of model
predictive control strategies for Micro Aerial Vehicles and specifically
multirotor configurations such as quadrotors. The diverse set of works in the
domain is organized based on the control law being optimized over linear or
nonlinear dynamics, the integration of state and input constraints, possible
fault-tolerant design, if reinforcement learning methods have been utilized and
if the controller refers to free-flight or other tasks such as physical
interaction or load transportation. A selected set of comparison results are
also presented and serve to provide insight for the selection between linear
and nonlinear schemes, the tuning of the prediction horizon, the importance of
disturbance observer-based offset-free tracking and the intrinsic robustness of
such methods to parameter uncertainty. Furthermore, an overview of recent
research trends on the combined application of modern deep reinforcement
learning techniques and model predictive control for multirotor vehicles is
presented. Finally, this review concludes with explicit discussion regarding
selected open-source software packages that deliver off-the-shelf model
predictive control functionality applicable to a wide variety of Micro Aerial
Vehicle configurations
A review of aerial manipulation of small-scale rotorcraft unmanned robotic systems
Small-scale rotorcraft unmanned robotic systems (SRURSs) are a kind of unmanned rotorcraft with manipulating devices. This review aims to provide an overview on aerial manipulation of SRURSs nowadays and promote relative research in the future. In the past decade, aerial manipulation of SRURSs has attracted the interest of researchers globally. This paper provides a literature review of the last 10 years (2008–2017) on SRURSs, and details achievements and challenges. Firstly, the definition, current state, development, classification, and challenges of SRURSs are introduced. Then, related papers are organized into two topical categories: mechanical structure design, and modeling and control. Following this, research groups involved in SRURS research and their major achievements are summarized and classified in the form of tables. The research groups are introduced in detail from seven parts. Finally, trends and challenges are compiled and presented to serve as a resource for researchers interested in aerial manipulation of SRURSs. The problem, trends, and challenges are described from three aspects. Conclusions of the paper are presented, and the future of SRURSs is discussed to enable further research interests
Agile load transportation systems using aerial robots
In this dissertation, we address problems that can occur during load transport using aerial robots, i.e., small scale quadrotors. First, detailed models of such transportation system are derived. These models include nonlinear models of a quadrotor, a model of a quadrotor carrying a fixed load and a model of a quadrotor carrying a suspended load. Second, the problem of quadrotor stabilization and trajectory tracking with changes of the center of gravity of the transportation system is addressed. This problem is solved using model reference adaptive control based on output feedback linearization that compensates for dynamical changes in the center of gravity of the quadrotor. The third problem we address is a problem of a swing-free transport of suspended load using quadrotors. Flying with a suspended load can be a very challenging and sometimes hazardous task as the suspended load significantly alters the flight characteristics of the quadrotor. In order to deal with suspended load flight, we present a method based on dynamic programming which is a model based offline method. The second investigated method we use is based on the Nelder-Mead algorithm which is an optimization technique used for nonlinear unconstrained optimization problems. This method is model free and it can be used for offline or online generation of the swing-free trajectories for the suspended load. Besides the swing-free maneuvers with suspended load, load trajectory tracking is another problem we solve in this dissertation. In order to solve this problem we use a Nelder-Mead based algorithm. In addition, we use an online least square policy iteration algorithm. At the end, we propose a high level algorithm for navigation in cluttered environments considering a quadrotor with suspended load. Furthermore, distributed control of multiple quadrotors with suspended load is addressed too. The proposed hierarchical architecture presented in this doctoral dissertation is an important step towards developing the next generation of agile autonomous aerial vehicles. These control algorithms enable quadrotors to display agile maneuvers while reconfiguring in real time whenever a change in the center of gravity occurs. This enables a swing-free load transport or trajectory tracking of the load in urban environments in a decentralized fashion
Adaptive and learning-based formation control of swarm robots
Autonomous aerial and wheeled mobile robots play a major role in tasks such as search and rescue, transportation, monitoring, and inspection. However, these operations are faced with a few open challenges including robust autonomy, and adaptive coordination based on the environment and operating conditions, particularly in swarm robots with limited communication and perception capabilities. Furthermore, the computational complexity increases exponentially with the number of robots in the swarm. This thesis examines two different aspects of the formation control problem. On the one hand, we investigate how formation could be performed by swarm robots with limited communication and perception (e.g., Crazyflie nano quadrotor). On the other hand, we explore human-swarm interaction (HSI) and different shared-control mechanisms between human and swarm robots (e.g., BristleBot) for artistic creation. In particular, we combine bio-inspired (i.e., flocking, foraging) techniques with learning-based control strategies (using artificial neural networks) for adaptive control of multi- robots. We first review how learning-based control and networked dynamical systems can be used to assign distributed and decentralized policies to individual robots such that the desired formation emerges from their collective behavior. We proceed by presenting a novel flocking control for UAV swarm using deep reinforcement learning. We formulate the flocking formation problem as a partially observable Markov decision process (POMDP), and consider a leader-follower configuration, where consensus among all UAVs is used to train a shared control policy, and each UAV performs actions based on the local information it collects. In addition, to avoid collision among UAVs and guarantee flocking and navigation, a reward function is added with the global flocking maintenance, mutual reward, and a collision penalty. We adapt deep deterministic policy gradient (DDPG) with centralized training and decentralized execution to obtain the flocking control policy using actor-critic networks and a global state space matrix. In the context of swarm robotics in arts, we investigate how the formation paradigm can serve as an interaction modality for artists to aesthetically utilize swarms. In particular, we explore particle swarm optimization (PSO) and random walk to control the communication between a team of robots with swarming behavior for musical creation
- …