Autonomous systems, such as self-driving cars and drones, have made
significant strides in recent years by leveraging visual inputs and machine
learning for decision-making and control. Despite their impressive performance,
these vision-based controllers can make erroneous predictions when faced with
novel or out-of-distribution inputs. Such errors can cascade to catastrophic
system failures and compromise system safety. In this work, we introduce a
run-time anomaly monitor to detect and mitigate such closed-loop, system-level
failures. Specifically, we leverage a reachability-based framework to
stress-test the vision-based controller offline and mine its system-level
failures. This data is then used to train a classifier that is leveraged online
to flag inputs that might cause system breakdowns. The anomaly detector
highlights issues that transcend individual modules and pertain to the safety
of the overall system. We also design a fallback controller that robustly
handles these detected anomalies to preserve system safety. We validate the
proposed approach on an autonomous aircraft taxiing system that uses a
vision-based controller for taxiing. Our results show the efficacy of the
proposed approach in identifying and handling system-level anomalies,
outperforming methods such as prediction error-based detection, and ensembling,
thereby enhancing the overall safety and robustness of autonomous systems