Ensuring the safety of autonomous vehicles (AVs) is of utmost importance and
testing them in simulated environments is a safer option than conducting
in-field operational tests. However, generating an exhaustive test suite to
identify critical test scenarios is computationally expensive as the
representation of each test is complex and contains various dynamic and static
features, such as the AV under test, road participants (vehicles, pedestrians,
and static obstacles), environmental factors (weather and light), and the
road's structural features (lanes, turns, road speed, etc.). In this paper, we
present a systematic technique that uses Instance Space Analysis (ISA) to
identify the significant features of test scenarios that affect their ability
to reveal the unsafe behaviour of AVs. ISA identifies the features that best
differentiate safety-critical scenarios from normal driving and visualises the
impact of these features on test scenario outcomes (safe/unsafe) in 2D. This
visualization helps to identify untested regions of the instance space and
provides an indicator of the quality of the test suite in terms of the
percentage of feature space covered by testing. To test the predictive ability
of the identified features, we train five Machine Learning classifiers to
classify test scenarios as safe or unsafe. The high precision, recall, and F1
scores indicate that our proposed approach is effective in predicting the
outcome of a test scenario without executing it and can be used for test
generation, selection, and prioritization.Comment: 28 pages, 6 figure