1 research outputs found
Automated System Performance Testing at MongoDB
Distributed Systems Infrastructure (DSI) is MongoDB's framework for running
fully automated system performance tests in our Continuous Integration (CI)
environment. To run in CI it needs to automate everything end-to-end:
provisioning and deploying multi-node clusters, executing tests, tuning the
system for repeatable results, and collecting and analyzing the results. Today
DSI is MongoDB's most used and most useful performance testing tool. It runs
almost 200 different benchmarks in daily CI, and we also use it for manual
performance investigations. As we can alert the responsible engineer in a
timely fashion, all but one of the major regressions were fixed before the
4.2.0 release. We are also able to catch net new improvements, of which DSI
caught 17. We open sourced DSI in March 2020.Comment: Author Preprint. Appearing in DBTest.io 202