2 research outputs found
Scheduling the NASA Deep Space Network with Deep Reinforcement Learning
With three complexes spread evenly across the Earth, NASA's Deep Space
Network (DSN) is the primary means of communications as well as a significant
scientific instrument for dozens of active missions around the world. A rapidly
rising number of spacecraft and increasingly complex scientific instruments
with higher bandwidth requirements have resulted in demand that exceeds the
network's capacity across its 12 antennae. The existing DSN scheduling process
operates on a rolling weekly basis and is time-consuming; for a given week,
generation of the final baseline schedule of spacecraft tracking passes takes
roughly 5 months from the initial requirements submission deadline, with
several weeks of peer-to-peer negotiations in between. This paper proposes a
deep reinforcement learning (RL) approach to generate candidate DSN schedules
from mission requests and spacecraft ephemeris data with demonstrated
capability to address real-world operational constraints. A deep RL agent is
developed that takes mission requests for a given week as input, and interacts
with a DSN scheduling environment to allocate tracks such that its reward
signal is maximized. A comparison is made between an agent trained using
Proximal Policy Optimization and its random, untrained counterpart. The results
represent a proof-of-concept that, given a well-shaped reward signal, a deep RL
agent can learn the complex heuristics used by experts to schedule the DSN. A
trained agent can potentially be used to generate candidate schedules to
bootstrap the scheduling process and thus reduce the turnaround cycle for DSN
scheduling
Smart Scheduling based on Deep Reinforcement Learning for Cellular Networks
To improve the system performance towards the Shannon limit, advanced radio
resource management mechanisms play a fundamental role. In particular,
scheduling should receive much attention, because it allocates radio resources
among different users in terms of their channel conditions and QoS
requirements. The difficulties of scheduling algorithms are the tradeoffs need
to be made among multiple objectives, such as throughput, fairness and packet
drop rate. We propose a smart scheduling scheme based on deep reinforcement
learning (DRL). We not only verify the performance gain achieved, but also
provide implementation-friend designs, i.e., a scalable neural network design
for the agent and a virtual environment training framework. With the scalable
neural network design, the DRL agent can easily handle the cases when the
number of active users is time-varying without the need to redesign and retrain
the DRL agent. Training the DRL agent in a virtual environment offline first
and using it as the initial version in the practical usage helps to prevent the
system from suffering from performance and robustness degradation due to the
time-consuming training. Through both simulations and field tests, we show that
the DRL-based smart scheduling outperforms the conventional scheduling method
and can be adopted in practical systems.Comment: 14 figures, submitted to a journal for possible publicatio