In this paper, we~present a novel scheduling solution for a class of
System-on-Chip (SoC) systems where heterogeneous chip resources (DSP, FPGA,
GPU, etc.) must be efficiently scheduled for continuously arriving hierarchical
jobs with their tasks represented by a directed acyclic graph. Traditionally,
heuristic algorithms have been widely used for many resource scheduling
domains, and Heterogeneous Earliest Finish Time (HEFT) has been a dominating
state-of-the-art technique across a broad range of heterogeneous resource
scheduling domains over many years. Despite their long-standing popularity,
HEFT-like algorithms are known to be vulnerable to a small amount of noise
added to the environment. Our Deep Reinforcement Learning (DRL)-based SoC
Scheduler (DeepSoCS), capable of learning the "best" task ordering under
dynamic environment changes, overcomes the brittleness of rule-based schedulers
such as HEFT with significantly higher performance across different types of
jobs. We~describe a DeepSoCS design process using a real-time heterogeneous SoC
scheduling emulator, discuss major challenges, and present two novel neural
network design features that lead to outperforming HEFT: (i) hierarchical job-
and task-graph embedding; and (ii) efficient use of real-time task information
in the state space. Furthermore, we~introduce effective techniques to address
two fundamental challenges present in our environment: delayed consequences and
joint actions. Through an extensive simulation study, we~show that our DeepSoCS
exhibits the significantly higher performance of job execution time than that
of HEFT with a higher level of robustness under realistic noise conditions.
We~conclude with a discussion of the potential improvements for our DeepSoCS
neural scheduler.Comment: 18 pages, Accepted by Electronics 202
