Reliable Provisioning of Spot Instances for Compute-intensive Applications

Cloud computing providers are now offering their unused resources for leasing in the spot market, which has been considered the first step towards a full-fledged market economy for computational resources. Spot instances are virtual machines (VMs) available at lower prices than their standard on-demand counterparts. These VMs will run for as long as the current price is lower than the maximum bid price users are willing to pay per hour. Spot instances have been increasingly used for executing compute-intensive applications. In spite of an apparent economical advantage, due to an intermittent nature of biddable resources, application execution times may be prolonged or they may not finish at all. This paper proposes a resource allocation strategy that addresses the problem of running compute-intensive jobs on a pool of intermittent virtual machines, while also aiming to run applications in a fast and economical way. To mitigate potential unavailability periods, a multifaceted fault-aware resource provisioning policy is proposed. Our solution employs price and runtime estimation mechanisms, as well as three fault tolerance techniques, namely checkpointing, task duplication and migration. We evaluate our strategies using trace-driven simulations, which take as input real price variation traces, as well as an application trace from the Parallel Workload Archive. Our results demonstrate the effectiveness of executing applications on spot instances, respecting QoS constraints, despite occasional failures.Comment: 8 pages, 4 figure

Cluster-Wide Context Switch of Virtualized Jobs

International audienceClusters are mostly used through Resources Management Systems (RMS) with a static allocation of resources for a bounded amount of time. Those approaches are known to be insufficient for an efficient use of clusters. To provide a finer RMS, job preemption, migration and dynamic allocation of resources are required. However due to the complexity of developing and using such mechanisms, advanced scheduling strategies have rarely been deployed. This trend is currently evolving thanks to the use of migration and preemption capabilities of Virtual Machines (VMs). However, although the manipulation of jobs composed of VM enables to change the state of the jobs according to the scheduling objective, changing the state and the location of numerous VMs at each decision is tedious and degrades the overall performance. In addition to the scheduling policy implementation, developers have to focus on the feasibility of the actions while executing them in the most efficient way. In this paper, we argue such an operation is independent from the policy itself and can be addressed through a generic mechanism, the cluster-wide context switch. Thanks to it, developers can implement sophisticated algorithms to schedule jobs without handling the issues related to their manipulations. They only focus on the implementation of their algorithm to select the jobs to run while the cluster-wide context switch system performs the necessary actions to switch from the current to the new situation. As a proof of concept, we evaluate the interest of the cluster-wide context switch through a sample scheduler that executes jobs as early as possible, even partially, regarding to their current resources requirements and their priority

Running user-provided virtual machines in batch-oriented computing clusters

The use of virtualization in HPC clusters can provide rich software environments, application isolation and efficient workload management mechanisms, but system-level virtualization introduces a software layer on the computing nodes that reduces performance and inhibits the direct use of hardware devices. We propose an unobtrusive user-level platform that allows the execution of virtual machines inside batch jobs without limiting the computing cluster’s ability to execute the most demanding applications. A per-user platform uses a static mode in which the VMs run entirely using the resources of a single batch job and a dynamic mode in which the VMs navigate at runtime between the continuously allocated jobs node time-slots. A dynamic mode is introduced to build complex scenarios with several VMs for personalized HPC environments or persistent services such as databases or web services based applications. Fault-tolerant system agents, integrated using group communication primitives, control the system and execute user commands and automatic scheduling decisions made by an optional monitoring function. The performance of compute intensive applications running on our system suffers negligible overhead compared to the native configuration. The performance of distributed applications is dependent on their communication patterns as the user-mode network overlay introduces a relevant communication overhead.FC

Hybrid Cloud for Educational Sector

AbstractCloud is an emergent field that relates virtual memory and storage to real time application. The organization and architecture of cloud varies on the basis of its set up and uses. It has profound applications in the field ranging from education sector to social networking and extended to international relations and business. To evolve and instrument a cloud architecture by modifying the platforms and implementing the concept of hybrid cloud along with cloud building techniques, using Big Data Analytics to formulate the statistical organization of the cloud service. In this paper we propose the hybrid cloud interface where the user from different educational sectors interact with the cloud environment and request the cloud provider to showcase the resource

Court execution of the lease agreement

Purpose - This study aimed to determine the provisions of Article 1576 Civil Code that states that tenants right are not interrupted because of the sale and purchase reviewed from legal theory (material rights and individuals) and justice theory. Considering the judge decision (execution) on the request is most important part in settling case process, because the execution not only ends a dispute, but also provides a picture of legal certainty and rights protection for the public on the sale of goods leased third party. Methodology - This study uses normative legal approach (normative juridical) with the study object in the form of positive legal norms, principles and legal theory derived from secondary data.This study describes norm by examining the problems and solutions in accordance with the sense of justice and practical needs.Findings - The results show that views from the legal theory (material rights and individuals) and justice theory, the applicant of the execution is more entitled to the leased goods than the tenant, because the execution applicant has a material rights that are absolute and inherent is higher level than tenants rights are individual and relative, so it is fairer if the applicant’s right to execution should take precedence over the tenant’s righ

Virtual machine scheduling in dedicated computing clusters

Time-critical applications process a continuous stream of input data and have to meet specific timing constraints. A common approach to ensure that such an application satisfies its constraints is over-provisioning: The application is deployed in a dedicated cluster environment with enough processing power to achieve the target performance for every specified data input rate. This approach comes with a drawback: At times of decreased data input rates, the cluster resources are not fully utilized. A typical use case is the HLT-Chain application that processes physics data at runtime of the ALICE experiment at CERN. From a perspective of cost and efficiency it is desirable to exploit temporarily unused cluster resources. Existing approaches aim for that goal by running additional applications. These approaches, however, a) lack in flexibility to dynamically grant the time-critical application the resources it needs, b) are insufficient for isolating the time-critical application from harmful side-effects introduced by additional applications or c) are not general because application-specific interfaces are used. In this thesis, a software framework is presented that allows to exploit unused resources in a dedicated cluster without harming a time-critical application. Additional applications are hosted in Virtual Machines (VMs) and unused cluster resources are allocated to these VMs at runtime. In order to avoid resource bottlenecks, the resource usage of VMs is dynamically modified according to the needs of the time-critical application. For this purpose, a number of previously not combined methods is used. On a global level, appropriate VM manipulations like hot migration, suspend/resume and start/stop are determined by an informed search heuristic and applied at runtime. Locally on cluster nodes, a feedback-controlled adaption of VM resource usage is carried out in a decentralized manner. The employment of this framework allows to increase a cluster’s usage by running additional applications, while at the same time preventing negative impact towards a time-critical application. This capability of the framework is shown for the HLT-Chain application: In an empirical evaluation the cluster CPU usage is increased from 49% to 79%, additional results are computed and no negative effect towards the HLT-Chain application are observed

Virtual machine scheduling in dedicated computing clusters

Time-critical applications process a continuous stream of input data and have to meet specific timing constraints. A common approach to ensure that such an application satisfies its constraints is over-provisioning: The application is deployed in a dedicated cluster environment with enough processing power to achieve the target performance for every specified data input rate. This approach comes with a drawback: At times of decreased data input rates, the cluster resources are not fully utilized. A typical use case is the HLT-Chain application that processes physics data at runtime of the ALICE experiment at CERN. From a perspective of cost and efficiency it is desirable to exploit temporarily unused cluster resources. Existing approaches aim for that goal by running additional applications. These approaches, however, a) lack in flexibility to dynamically grant the time-critical application the resources it needs, b) are insufficient for isolating the time-critical application from harmful side-effects introduced by additional applications or c) are not general because application-specific interfaces are used. In this thesis, a software framework is presented that allows to exploit unused resources in a dedicated cluster without harming a time-critical application. Additional applications are hosted in Virtual Machines (VMs) and unused cluster resources are allocated to these VMs at runtime. In order to avoid resource bottlenecks, the resource usage of VMs is dynamically modified according to the needs of the time-critical application. For this purpose, a number of previously not combined methods is used. On a global level, appropriate VM manipulations like hot migration, suspend/resume and start/stop are determined by an informed search heuristic and applied at runtime. Locally on cluster nodes, a feedback-controlled adaption of VM resource usage is carried out in a decentralized manner. The employment of this framework allows to increase a cluster’s usage by running additional applications, while at the same time preventing negative impact towards a time-critical application. This capability of the framework is shown for the HLT-Chain application: In an empirical evaluation the cluster CPU usage is increased from 49% to 79%, additional results are computed and no negative effect towards the HLT-Chain application are observed

Scheduling Adaptability Experiments for Scientific Applications on Science Cloud

클라우드 컴퓨팅 환경에서 가상 머신을 기반으로 스케줄링 하는 것은 IaaS(Infrastructureas-a-service)로써 사용자가 원하는 가상 환경을 만들어 필요에 따라 서비스로 제공해 줄 수 있다는 이점을 가진다. 하지만 자원을 필요로 하는 여러 가지 작업이 동시에 이루어져야 할 때에 스케줄링의 불편과 자원의 비효율적 사용을 초래할 수 있다. 따라서 작업을 포함한 가상 머신들을 사용자의 요청에 따라 적절한 시간에 스케줄링 해 주는 것이 필요하다. 본 논문에서는 사이언스 클라우드 환경을 구축할 수 있는 오픈네블라(OpenNebula)[1]와 함께 Haizea 스케줄러[2]를 사용하여 사용자들의 작업 요청을 스케줄링 하는 것을 보인다. Haizea 스케줄러는 작업을 포함한 가상머신들의 우선순위와 자원의 활용도를 기반으로 가상머신을 보류/재시작(suspend/resume)하여 스케줄링 한다. 이는 전체 실행 시간을 줄이고 높은 우선순위를 가지는 작업을 포함하는 가상머신을 먼저 실행할 수 있도록 한다. 대규모 컴퓨팅자원을 필요로 하고 실행시간이 상대적으로 긴 과학 계산 어플리케이션을 대상으로 우선순위에 따라 가상머신을 실행하여 과학계산 클라우드 환경의 적응성을 실험하였다.OAIID:oai:osos.snu.ac.kr:snu2010-01/104/0000004648/26SEQ:26PERF_CD:SNU2010-01EVAL_ITEM_CD:104USER_ID:0000004648ADJUST_YN:NEMP_ID:A001138DEPT_CD:446CITE_RATE:0FILENAME:사이언스 클라우드 기반 과학 응용 스케줄링 적응성 실험.pdfDEPT_NM:기계항공공학부EMAIL:[email protected]: