Researchers and engineers are increasingly adopting cloud-native technologies
for application development and performance evaluation. While this has improved
the reproducibility of benchmarks in the cloud, the complexity of cloud-native
environments makes it difficult to run benchmarks reliably. Cloud-native
applications are often instrumented or altered at runtime, by dynamically
patching or hooking them, which introduces a significant performance overhead.
Our work discusses the benchmarking-related pitfalls of the dominant
cloud-native technology, Kubernetes, and how they affect performance
measurements of dynamically patched or hooked applications. We present
recommendations to mitigate these risks and demonstrate how an improper
experimental setup can negatively impact latency measurements.Comment: to be published in the 14th Symposium on Software Performance (SSP
2023), source code available at
https://github.com/dynatrace-research/function-hook-latency-benchmarkin