We develop DEEPTRAVERSAL, a feedback-driven framework to test DNNs.
DEEPTRAVERSAL first launches an offline phase to map media data of various
forms to manifolds. Then, in its online testing phase, DEEPTRAVERSAL traverses
the prepared manifold space to maximize DNN coverage criteria and trigger
prediction errors. In our evaluation, DNNs executing various tasks (e.g.,
classification, self-driving, machine translation) and media data of different
types (image, audio, text) were used. DEEPTRAVERSAL exhibits better performance
than prior methods with respect to popular DNN coverage criteria and it can
discover a larger number and higher quality of error-triggering inputs. The
tested DNN models, after being repaired with findings of DEEPTRAVERSAL, achieve
better accurac