Towards an MPI-like Framework for Azure Cloud Platform

Message passing interface (MPI) has been widely used for implementing parallel and distributed applications. The emergence of cloud computing offers a scalable, fault-tolerant, on-demand al-ternative to traditional on-premise clusters. In this thesis, we investigate the possibility of adopt-ing the cloud platform as an alternative to conventional MPI-based solutions. We show that cloud platform can exhibit competitive performance and benefit the users of this platform with its fault-tolerant architecture and on-demand access for a robust solution. Extensive research is done to identify the difficulties of designing and implementing an MPI-like framework for Azure cloud platform. We present the details of the key components required for implementing such a framework along with our experimental results for benchmarking multiple basic operations of MPI standard implemented in the cloud and its practical application in solving well-known large-scale algorithmic problems

Empirical Evaluation of Cloud IAAS Platforms using System-level Benchmarks

Cloud Computing is an emerging paradigm in the field of computing where scalable IT enabled capabilities are delivered ‘as-a-service’ using Internet technology. The Cloud industry adopted three basic types of computing service models based on software level abstraction: Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS), and Software-as-a-Service (SaaS). Infrastructure-as-a-Service allows customers to outsource fundamental computing resources such as servers, networking, storage, as well as services where the provider owns and manages the entire infrastructure. This allows customers to only pay for the resources they consume. In a fast-growing IaaS market with multiple cloud platforms offering IaaS services, the user\u27s decision on the selection of the best IaaS platform is quite challenging. Therefore, it is very important for organizations to evaluate and compare the performance of different IaaS cloud platforms in order to minimize cost and maximize performance. Using a vendor-neutral approach, this research focused on four of the top IaaS cloud platforms- Amazon EC2, Microsoft Azure, Google Compute Engine, and Rackspace cloud services. This research compared the performance of IaaS cloud platforms using system-level parameters including server, file I/O, and network. System-level benchmarking provides an objective comparison of the IaaS cloud platforms from performance perspective. Unixbench, Dbench, and Iperf are the system-level benchmarks chosen to test the performance of the server, file I/O, and network respectively. In order to capture the performance variability, the benchmark tests were performed at different time periods on weekdays and weekends. Each IaaS platform\u27s performance was also tested using various parameters. The benchmark tests conducted on different virtual machine (VM) configurations should help cloud users select the best IaaS platform for their needs. Also, based on their applications\u27 requirements, cloud users should get a clearer picture of which VM configuration they should choose. In addition to the performance evaluation, the price-per-performance value of all the IaaS cloud platforms was also examined

Hybrid High Performance Computing (HPC) + Cloud for Scientific Computing

The HPC+Cloud framework has been built to enable on-premise HPC jobs to use resources from cloud computing nodes. As part of designing the software framework, public cloud providers, namely Amazon AWS, Microsoft Azure and NeCTAR were benchmarked against one another, and Microsoft Azure was determined to be the most suitable cloud component in the proposed HPC+Cloud software framework. Finally, an HPC+Cloud cluster was built using the HPC+Cloud software framework and then was validated by conducting HPC processing benchmarks

Scientific High Performance Computing (HPC) Applications On The Azure Cloud Platform

Cloud computing is emerging as a promising platform for compute and data intensive scientific applications. Thanks to the on-demand elastic provisioning capabilities, cloud computing has instigated curiosity among researchers from a wide range of disciplines. However, even though many vendors have rolled out their commercial cloud infrastructures, the service offerings are usually only best-effort based without any performance guarantees. Utilization of these resources will be questionable if it can not meet the performance expectations of deployed applications. Additionally, the lack of the familiar development tools hamper the productivity of eScience developers to write robust scientific high performance computing (HPC) applications. There are no standard frameworks that are currently supported by any large set of vendors offering cloud computing services. Consequently, the application portability among different cloud platforms for scientific applications is hard. Among all clouds, the emerging Azure cloud from Microsoft in particular remains a challenge for HPC program development both due to lack of its support for traditional parallel programming support such as Message Passing Interface (MPI) and map-reduce and due to its evolving application programming interfaces (APIs). We have designed newer frameworks and runtime environments to help HPC application developers by providing them with easy to use tools similar to those known from traditional parallel and distributed computing environment set- ting, such as MPI, for scientific application development on the Azure cloud platform. It is challenging to create an efficient framework for any cloud platform, including the Windows Azure platform, as they are mostly offered to users as a black-box with a set of application programming interfaces (APIs) to access various service components. The primary contributions of this Ph.D. thesis are (i) creating a generic framework for bag-of-tasks HPC applications to serve as the basic building block for application development on the Azure cloud platform, (ii) creating a set of APIs for HPC application development over the Azure cloud platform, which is similar to message passing interface (MPI) from traditional parallel and distributed setting, and (iii) implementing Crayons using the proposed APIs as the first end-to-end parallel scientific application to parallelize the fundamental GIS operations

Mobile Cloud Computing: Offloading Mobile Processing to the Cloud

The current proliferation of mobile systems, such as smart phones, PDA and tablets, has led to their adoption as the primary computing platforms for many users. This trend suggests that designers will continue to aim towards the convergence of functionality on a single mobile device. However, this convergence penalizes the mobile system in computational resources such as processor speed, memory consumption, disk capacity, as well as in weight, size, ergonomics and the user’s most important component, battery life. Therefore, this current trend aims towards the efficient and effective use of its hardware and software components. Hence, energy consumption and response time are major concerns when executing complex algorithms on mobile devices because they require significant resources to solve intricate problems. Current cloud computing environments for performing complex and data intensive computation remotely are likely to be an excellent solution for off-loading computation and data processing from mobile devices restricted by reduced resources. In cloud computing, virtualization enables a logical abstraction of physical components in a scalable manner that can overcome the physical constraint of resources. This optimizes IT infrastructure and makes cloud computing a worthy cost effective solution. The intent of this thesis is to determine the types of applications that are better suited to be off-loaded to the cloud from mobile devices. To this end, this thesis quantitatively and qualitatively compares the performance of executing two different kinds of workloads locally on two different mobile devices and remotely on two different cloud computing providers. The results of this thesis are expected to provide valuable insight to developers and architects of mobile applications by providing information on the applications that can be performed remotely in order to save energy and get better response times while remaining transparent to users

Reusable generic software robot

Abstract. The main purpose of this thesis was to create a generic reusable software robot which can be deployed into any IaaS type of a cloud service. In this thesis the first thing to be researched was how to implement a virtualised environment into a cloud service. The possibilities for virtualising the environment were a container and a virtual machine. The two possible implementations were researched since the resulting implementation must be compatible with a cloud service. Firstly, it was found that a container-based implementation would be the best option because it is lightweight to move around and secondly, a start-up time of a new instance in a cloud service is fast. Possible cloud providers were scanned after researching possible implementation methods. Two possible cloud providers, AWS and Azure, were studied more closely since they offer an infrastructure as a service and once they are commonly used. AWS was chosen to be the platform to be used because of a higher maturity level and also because of the possibility to add or remove container capabilities. Finally, it was discussed how a generic reusable software robot was implemented. Notable circumstances of suitable tasks for a software robot were considered.Kertakäyttöinen geneerinen ohjelmistorobotti. Tiivistelmä. Tässä työssä tutkittiin, kuinka geneerinen kertakäyttöinen ohjelmistorobotti voidaan toteuttaa pilvipalvelussa. Ensin tarkasteltiin erilaisia virtualisointimenetelmiä, joilla ohjelmistorobotti voitaisiin toteuttaa. Tutkitut menetelmät olivat virtuaalikone ja kontti. Näitä kahta toteutustapaa vertailtiin huomioiden valmiin toteutuksen sopivuus pilvipalveluun. Kontti todettiin sopivimmaksi toteutustavaksi, koska se vie vähän tilaa ja uuden instanssin käynnistäminen on nopeaa. Pilvipalvelutarjoajia tutkittiin, kun sopiva toteutusmenetelmä ohjelmistorobotille oli löydetty. Tutkimuksessa keskityttiin AWS:ään ja Azureen, jotka ovat tällä hetkellä suurimpia markkinoilla toimivia infrastructure as a service -tyyppisten pilvipalveuiden tarjoajia. AWS valittiin toteutusalustaksi, koska se on teknisesti edistyneempi kuin Azure ja AWS:ssä on mahdollista lisätä ja poistaa kontin oikeuksia. Lopuksi esiteltiin, kuinka geneerinen kertakäyttöinen ohjelmistorobotti toteutettiin ja mitä täytyy ottaa huomioon, kun päätetään sopivasta käyttökohteesta ohjelmistorobotille

FaaSdom: A Benchmark Suite for Serverless Computing

Serverless computing has become a major trend among cloud providers. With serverless computing, developers fully delegate the task of managing the servers, dynamically allocating the required resources, as well as handling availability and fault-tolerance matters to the cloud provider. In doing so, developers can solely focus on the application logic of their software, which is then deployed and completely managed in the cloud. Despite its increasing popularity, not much is known regarding the actual system performance achievable on the currently available serverless platforms. Specifically, it is cumbersome to benchmark such systems in a language- or runtime-independent manner. Instead, one must resort to a full application deployment, to later take informed decisions on the most convenient solution along several dimensions, including performance and economic costs. FaaSdom is a modular architecture and proof-of-concept implementation of a benchmark suite for serverless computing platforms. It currently supports the current mainstream serverless cloud providers (i.e., AWS, Azure, Google, IBM), a large set of benchmark tests and a variety of implementation languages. The suite fully automatizes the deployment, execution and clean-up of such tests, providing insights (including historical) on the performance observed by serverless applications. FaaSdom also integrates a model to estimate budget costs for deployments across the supported providers. FaaSdom is open-source and available at https://github.com/bschitter/benchmark-suite-serverless-computing.Comment: ACM DEBS'2