Cloud Computing, as one of the most promising computing paradigms, has become
increasingly accepted in industry. Numerous commercial providers have started
to supply public Cloud services, and corresponding performance evaluation is
then inevitably required for Cloud provider selection or cost-benefit analysis.
Unfortunately, inaccurate and confusing evaluation implementations can be often
seen in the context of commercial Cloud Computing, which could severely
interfere and spoil evaluation-related comprehension and communication. This
paper introduces a taxonomy to help profile and standardize the details of
performance evaluation of commercial Cloud services. Through a systematic
literature review, we constructed the taxonomy along two dimensions by
arranging the atomic elements of Cloud-related performance evaluation. As such,
this proposed taxonomy can be employed both to analyze existing evaluation
practices through decomposition into elements and to design new experiments
through composing elements for evaluating performance of commercial Cloud
services. Moreover, through smooth expansion, we can continually adapt this
taxonomy to the more general area of evaluation of Cloud Computing.Comment: 8 pages, Proceedings of the 5th International Conference on Cloud
Computing (IEEE CLOUD 2012), pp. 344-351, Honolulu, Hawaii, USA, June 24-29,
201