A Cost-Effective Strategy for Storing Scientific Datasets with Multiple Service Providers in the Cloud

Cloud computing provides scientists a platform that can deploy computation and data intensive applications without infrastructure investment. With excessive cloud resources and a decision support system, large generated data sets can be flexibly 1 stored locally in the current cloud, 2 deleted and regenerated whenever reused or 3 transferred to cheaper cloud service for storage. However, due to the pay for use model, the total application cost largely depends on the usage of computation, storage and bandwidth resources, hence cutting the cost of cloud based data storage becomes a big concern for deploying scientific applications in the cloud. In this paper, we propose a novel strategy that can cost effectively store large generated data sets with multiple cloud service providers. The strategy is based on a novel algorithm that finds the trade off among computation, storage and bandwidth costs in the cloud, which are three key factors for the cost of data storage. Both general (random) simulations conducted with popular cloud service providers pricing models and three specific case studies on real world scientific applications show that the proposed storage strategy is highly cost effective and practical for run time utilization in the cloud.Comment: 20 page

A Scalable Architecture for e-Science Data Management

Abstract—The massive increase in the size of the data provided by e-Science applications requires not only to increase the capabilities of resources, but also to design new strategies for efficient utilization of already available resources. In this paper we present a scalable approach to extend a file-oriented storage system, Chelonia, with geographically distributed databases defined by a generic database schema. The database schema is able to model the data from typical e-Science applications. The system includes web service query service allowing e-Science applications to query the required data. I