1 research outputs found
Learning Non-robustness using Simulation-based Testing: a Network Traffic-shaping Case Study
An input to a system reveals a non-robust behaviour when, by making a small
change in the input, the output of the system changes from acceptable (passing)
to unacceptable (failing) or vice versa. Identifying inputs that lead to
non-robust behaviours is important for many types of systems, e.g.,
cyber-physical and network systems, whose inputs are prone to perturbations. In
this paper, we propose an approach that combines simulation-based testing with
regression tree models to generate value ranges for inputs in response to which
a system is likely to exhibit non-robust behaviours. We apply our approach to a
network traffic-shaping system (NTSS) -- a novel case study from the network
domain. In this case study, developed and conducted in collaboration with a
network solutions provider, RabbitRun Technologies, input ranges that lead to
non-robustness are of interest as a way to identify and mitigate network
quality-of-service issues. We demonstrate that our approach accurately
characterizes non-robust test inputs of NTSS by achieving a precision of 84%
and a recall of 100%, significantly outperforming a standard baseline. In
addition, we show that there is no statistically significant difference between
the results obtained from our simulated testbed and a hardware testbed with
identical configurations. Finally we describe lessons learned from our
industrial collaboration, offering insights about how simulation helps discover
unknown and undocumented behaviours as well as a new perspective on using
non-robustness as a measure for system re-configuration.Comment: This paper is accepted at the 16th IEEE International Conference on
Software Testing, Verification and Validation (ICST 2023