11,540 research outputs found
Fault Tolerant Deep Reinforcement Learning for Aerospace Applications
With the growing use of Unmanned Aerial Systems, a new need has risen for intelligent algorithms that not only stabilize or control the system, but rather would also include various factors such as optimality, robustness, adaptability, tracking, decision making, and many more. In this thesis, a deep-learning-based control system is designed with fault-tolerant and disturbance rejection capabilities and applied to a high-order nonlinear dynamic system. The approach uses a Reinforcement Learning architecture that combines concepts from optimal control, robust control, and game theory to create an optimally adaptive control for disturbance rejection. Additionally, a cascaded Observer-based Kalman Filter is formulated for estimating adverse inputs to the system. Numerical simulations are presented using different nonlinear model dynamics and scenarios. The Deep Reinforcement Learning and Observer architecture is demonstrated to be a promising control system alternative for fault tolerant applications
Model Predictive Control for Micro Aerial Vehicles: A Survey
This paper presents a review of the design and application of model
predictive control strategies for Micro Aerial Vehicles and specifically
multirotor configurations such as quadrotors. The diverse set of works in the
domain is organized based on the control law being optimized over linear or
nonlinear dynamics, the integration of state and input constraints, possible
fault-tolerant design, if reinforcement learning methods have been utilized and
if the controller refers to free-flight or other tasks such as physical
interaction or load transportation. A selected set of comparison results are
also presented and serve to provide insight for the selection between linear
and nonlinear schemes, the tuning of the prediction horizon, the importance of
disturbance observer-based offset-free tracking and the intrinsic robustness of
such methods to parameter uncertainty. Furthermore, an overview of recent
research trends on the combined application of modern deep reinforcement
learning techniques and model predictive control for multirotor vehicles is
presented. Finally, this review concludes with explicit discussion regarding
selected open-source software packages that deliver off-the-shelf model
predictive control functionality applicable to a wide variety of Micro Aerial
Vehicle configurations
Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications
Wireless sensor networks monitor dynamic environments that change rapidly
over time. This dynamic behavior is either caused by external factors or
initiated by the system designers themselves. To adapt to such conditions,
sensor networks often adopt machine learning techniques to eliminate the need
for unnecessary redesign. Machine learning also inspires many practical
solutions that maximize resource utilization and prolong the lifespan of the
network. In this paper, we present an extensive literature review over the
period 2002-2013 of machine learning methods that were used to address common
issues in wireless sensor networks (WSNs). The advantages and disadvantages of
each proposed algorithm are evaluated against the corresponding problem. We
also provide a comparative guide to aid WSN designers in developing suitable
machine learning solutions for their specific application challenges.Comment: Accepted for publication in IEEE Communications Surveys and Tutorial
Towards real-time reinforcement learning control of a wave energy converter
The levellised cost of energy of wave energy converters (WECs) is not competitive with fossil fuel-powered stations yet. To improve the feasibility of wave energy, it is necessary to develop effective control strategies that maximise energy absorption in mild sea states, whilst limiting motions in high waves. Due to their model-based nature, state-of-the-art control schemes struggle to deal with model uncertainties, adapt to changes in the system dynamics with time, and provide real-time centralised control for large arrays of WECs. Here, an alternative solution is introduced to address these challenges, applying deep reinforcement learning (DRL) to the control of WECs for the first time. A DRL agent is initialised from data collected in multiple sea states under linear model predictive control in a linear simulation environment. The agent outperforms model predictive control for high wave heights and periods, but suffers close to the resonant period of the WEC. The computational cost at deployment time of DRL is also much lower by diverting the computational effort from deployment time to training. This provides confidence in the application of DRL to large arrays of WECs, enabling economies of scale. Additionally, model-free reinforcement learning can autonomously adapt to changes in the system dynamics, enabling fault-tolerant control
Fast Damage Recovery in Robotics with the T-Resilience Algorithm
Damage recovery is critical for autonomous robots that need to operate for a
long time without assistance. Most current methods are complex and costly
because they require anticipating each potential damage in order to have a
contingency plan ready. As an alternative, we introduce the T-resilience
algorithm, a new algorithm that allows robots to quickly and autonomously
discover compensatory behaviors in unanticipated situations. This algorithm
equips the robot with a self-model and discovers new behaviors by learning to
avoid those that perform differently in the self-model and in reality. Our
algorithm thus does not identify the damaged parts but it implicitly searches
for efficient behaviors that do not use them. We evaluate the T-Resilience
algorithm on a hexapod robot that needs to adapt to leg removal, broken legs
and motor failures; we compare it to stochastic local search, policy gradient
and the self-modeling algorithm proposed by Bongard et al. The behavior of the
robot is assessed on-board thanks to a RGB-D sensor and a SLAM algorithm. Using
only 25 tests on the robot and an overall running time of 20 minutes,
T-Resilience consistently leads to substantially better results than the other
approaches
- …