On a Catalogue of Metrics for Evaluating Commercial Cloud Services

Given the continually increasing amount of commercial Cloud services in the market, evaluation of different services plays a significant role in cost-benefit analysis or decision making for choosing Cloud Computing. In particular, employing suitable metrics is essential in evaluation implementations. However, to the best of our knowledge, there is not any systematic discussion about metrics for evaluating Cloud services. By using the method of Systematic Literature Review (SLR), we have collected the de facto metrics adopted in the existing Cloud services evaluation work. The collected metrics were arranged following different Cloud service features to be evaluated, which essentially constructed an evaluation metrics catalogue, as shown in this paper. This metrics catalogue can be used to facilitate the future practice and research in the area of Cloud services evaluation. Moreover, considering metrics selection is a prerequisite of benchmark selection in evaluation implementations, this work also supplements the existing research in benchmarking the commercial Cloud services.Comment: 10 pages, Proceedings of the 13th ACM/IEEE International Conference on Grid Computing (Grid 2012), pp. 164-173, Beijing, China, September 20-23, 201

Building an Expert System for Evaluation of Commercial Cloud Services

Commercial Cloud services have been increasingly supplied to customers in industry. To facilitate customers' decision makings like cost-benefit analysis or Cloud provider selection, evaluation of those Cloud services are becoming more and more crucial. However, compared with evaluation of traditional computing systems, more challenges will inevitably appear when evaluating rapidly-changing and user-uncontrollable commercial Cloud services. This paper proposes an expert system for Cloud evaluation that addresses emerging evaluation challenges in the context of Cloud Computing. Based on the knowledge and data accumulated by exploring the existing evaluation work, this expert system has been conceptually validated to be able to give suggestions and guidelines for implementing new evaluation experiments. As such, users can conveniently obtain evaluation experiences by using this expert system, which is essentially able to make existing efforts in Cloud services evaluation reusable and sustainable.Comment: 8 page, Proceedings of the 2012 International Conference on Cloud and Service Computing (CSC 2012), pp. 168-175, Shanghai, China, November 22-24, 201

A Factor Framework for Experimental Design for Performance Evaluation of Commercial Cloud Services

Given the diversity of commercial Cloud services, performance evaluations of candidate services would be crucial and beneficial for both service customers (e.g. cost-benefit analysis) and providers (e.g. direction of service improvement). Before an evaluation implementation, the selection of suitable factors (also called parameters or variables) plays a prerequisite role in designing evaluation experiments. However, there seems a lack of systematic approaches to factor selection for Cloud services performance evaluation. In other words, evaluators randomly and intuitively concerned experimental factors in most of the existing evaluation studies. Based on our previous taxonomy and modeling work, this paper proposes a factor framework for experimental design for performance evaluation of commercial Cloud services. This framework capsules the state-of-the-practice of performance evaluation factors that people currently take into account in the Cloud Computing domain, and in turn can help facilitate designing new experiments for evaluating Cloud services.Comment: 8 pages, Proceedings of the 4th International Conference on Cloud Computing Technology and Science (CloudCom 2012), pp. 169-176, Taipei, Taiwan, December 03-06, 201

Towards a Taxonomy of Performance Evaluation of Commercial Cloud Services

Cloud Computing, as one of the most promising computing paradigms, has become increasingly accepted in industry. Numerous commercial providers have started to supply public Cloud services, and corresponding performance evaluation is then inevitably required for Cloud provider selection or cost-benefit analysis. Unfortunately, inaccurate and confusing evaluation implementations can be often seen in the context of commercial Cloud Computing, which could severely interfere and spoil evaluation-related comprehension and communication. This paper introduces a taxonomy to help profile and standardize the details of performance evaluation of commercial Cloud services. Through a systematic literature review, we constructed the taxonomy along two dimensions by arranging the atomic elements of Cloud-related performance evaluation. As such, this proposed taxonomy can be employed both to analyze existing evaluation practices through decomposition into elements and to design new experiments through composing elements for evaluating performance of commercial Cloud services. Moreover, through smooth expansion, we can continually adapt this taxonomy to the more general area of evaluation of Cloud Computing.Comment: 8 pages, Proceedings of the 5th International Conference on Cloud Computing (IEEE CLOUD 2012), pp. 344-351, Honolulu, Hawaii, USA, June 24-29, 201

On Evaluating Commercial Cloud Services: A Systematic Review

Background: Cloud Computing is increasingly booming in industry with many competing providers and services. Accordingly, evaluation of commercial Cloud services is necessary. However, the existing evaluation studies are relatively chaotic. There exists tremendous confusion and gap between practices and theory about Cloud services evaluation. Aim: To facilitate relieving the aforementioned chaos, this work aims to synthesize the existing evaluation implementations to outline the state-of-the-practice and also identify research opportunities in Cloud services evaluation. Method: Based on a conceptual evaluation model comprising six steps, the Systematic Literature Review (SLR) method was employed to collect relevant evidence to investigate the Cloud services evaluation step by step. Results: This SLR identified 82 relevant evaluation studies. The overall data collected from these studies essentially represent the current practical landscape of implementing Cloud services evaluation, and in turn can be reused to facilitate future evaluation work. Conclusions: Evaluation of commercial Cloud services has become a world-wide research topic. Some of the findings of this SLR identify several research gaps in the area of Cloud services evaluation (e.g., the Elasticity and Security evaluation of commercial Cloud services could be a long-term challenge), while some other findings suggest the trend of applying commercial Cloud services (e.g., compared with PaaS, IaaS seems more suitable for customers and is particularly important in industry). This SLR study itself also confirms some previous experiences and reveals new Evidence-Based Software Engineering (EBSE) lessons

Quantum Emission From Hexagonal Boron Nitride Monolayers

Atomically thin van der Waals crystals have recently enabled new scientific and technological breakthroughs across a variety of disciplines in materials science, nanophotonics and physics. However, non-classical photon emission from these materials has not been achieved to date. Here we report room temperature quantum emission from hexagonal boron nitride nanoflakes. The single photon emitter exhibits a combination of superb quantum optical properties at room temperature that include the highest brightness reported in the visible part of the spectrum, narrow line width, absolute photo-stability, a short excited state lifetime and a high quantum efficiency. Density functional theory modeling suggests that the emitter is the antisite nitrogen vacancy defect that is present in single and multi-layer hexagonal boron nitride. Our results constitute the unprecedented potential of van der Waals crystals for nanophotonics, optoelectronics and quantum information processing