11 research outputs found
UAV Path Planning for Wireless Data Harvesting: A Deep Reinforcement Learning Approach
Autonomous deployment of unmanned aerial vehicles (UAVs) supporting
next-generation communication networks requires efficient trajectory planning
methods. We propose a new end-to-end reinforcement learning (RL) approach to
UAV-enabled data collection from Internet of Things (IoT) devices in an urban
environment. An autonomous drone is tasked with gathering data from distributed
sensor nodes subject to limited flying time and obstacle avoidance. While
previous approaches, learning and non-learning based, must perform expensive
recomputations or relearn a behavior when important scenario parameters such as
the number of sensors, sensor positions, or maximum flying time, change, we
train a double deep Q-network (DDQN) with combined experience replay to learn a
UAV control policy that generalizes over changing scenario parameters. By
exploiting a multi-layer map of the environment fed through convolutional
network layers to the agent, we show that our proposed network architecture
enables the agent to make movement decisions for a variety of scenario
parameters that balance the data collection goal with flight time efficiency
and safety constraints. Considerable advantages in learning efficiency from
using a map centered on the UAV's position over a non-centered map are also
illustrated.Comment: Code available under
https://github.com/hbayerlein/uav_data_harvesting, IEEE Global Communications
Conference (GLOBECOM) 202
Kinematic Model for Fixed-Wing Aircraft with Constrained Roll-Rate
The technical report derives a kinematic model of fixed-wing aircraft that is based on constrained roll rate. This new kinematic model can be used for trajectory planning and optimization.National Science Foundation (NSF), CNS-1646383Ope
Learning to Recharge: UAV Coverage Path Planning through Deep Reinforcement Learning
Coverage path planning (CPP) is a critical problem in robotics, where the
goal is to find an efficient path that covers every point in an area of
interest. This work addresses the power-constrained CPP problem with recharge
for battery-limited unmanned aerial vehicles (UAVs). In this problem, a notable
challenge emerges from integrating recharge journeys into the overall coverage
strategy, highlighting the intricate task of making strategic, long-term
decisions. We propose a novel proximal policy optimization (PPO)-based deep
reinforcement learning (DRL) approach with map-based observations, utilizing
action masking and discount factor scheduling to optimize coverage trajectories
over the entire mission horizon. We further provide the agent with a position
history to handle emergent state loops caused by the recharge capability. Our
approach outperforms a baseline heuristic, generalizes to different target
zones and maps, with limited generalization to unseen maps. We offer valuable
insights into DRL algorithm design for long-horizon problems and provide a
publicly available software framework for the CPP problem.Comment: This work has been submitted to the IEEE for possible publication.
Copyright may be transferred without notice, after which this version may no
longer be accessibl
Edge Generation Scheduling for DAG Tasks using Deep Reinforcement Learning
Directed acyclic graph (DAG) tasks are currently adopted in the real-time
domain to model complex applications from the automotive, avionics, and
industrial domain that implement their functionalities through chains of
intercommunicating tasks. This paper studies the problem of scheduling
real-time DAG tasks by presenting a novel schedulability test based on the
concept of trivial schedulability. Using this schedulability test, we propose a
new DAG scheduling framework (edge generation scheduling -- EGS) that attempts
to minimize the DAG width by iteratively generating edges while guaranteeing
the deadline constraint. We study how to efficiently solve the problem of
generating edges by developing a deep reinforcement learning algorithm combined
with a graph representation neural network to learn an efficient edge
generation policy for EGS. We evaluate the effectiveness of the proposed
algorithm by comparing it with state-of-the-art DAG scheduling heuristics and
an optimal mixed-integer linear programming baseline. Experimental results show
that the proposed algorithm outperforms the state-of-the-art by requiring fewer
processors to schedule the same DAG tasks.Comment: Under revie
A Cyber-Physical Prototyping and Testing Framework to Enable the Rapid Development of UAVs
In this work, a cyber-physical prototyping and testing framework to enable the rapid development of UAVs is conceived and demonstrated. The UAV Development Framework is an extension of the typical iterative engineering design and development process, specifically applied to the rapid development of UAVs. Unlike other development frameworks in the literature, the presented framework allows for iteration throughout the entire development process from design to construction, using a mixture of simulated and real-life testing as well as cross-aircraft development. The framework presented includes low- and high-order methods and tools that can be applied to a broad range of fixed-wing UAVs and can either be combined and executed simultaneously or be executed sequentially. As part of this work, seven novel and enhanced methods and tools were developed that apply to fixed-wing UAVs in the areas of: flight testing, measurement, modeling and emulation, and optimization. A demonstration of the framework to quickly develop an unmanned aircraft for agricultural field surveillance is presented