Systematic and recomputable comparison of multi-cloud management platforms

With the growth and evolution of cloud applications, more and more architectures use hybrid cloud bindings to optimally use virtual resources regarding pricing policies and performance. This process has led to the creation of multi-cloud management platforms as well as abstraction libraries. At the moment, many (multi-)cloud management platforms (CMPs) are designed to cover the functional requirements. Along with growing adoption and industrial impact of such solutions, there is a need for a comparison and test environment which automatically assesses and compares existing platforms and helps in choosing the optimal one. This paper focuses on the creation of a suitable testbed concept and an actual extensible software prototype which makes multi-cloud experiments repeatable and reusable by other researchers. The work is evaluated by an exemplary comparison of 4 CMPs bound to AWS, showcasing standardised output formats and evaluation criteria