Predicting Scheduling Failures in the Cloud

Cloud Computing has emerged as a key technology to deliver and manage computing, platform, and software services over the Internet. Task scheduling algorithms play an important role in the efficiency of cloud computing services as they aim to reduce the turnaround time of tasks and improve resource utilization. Several task scheduling algorithms have been proposed in the literature for cloud computing systems, the majority relying on the computational complexity of tasks and the distribution of resources. However, several tasks scheduled following these algorithms still fail because of unforeseen changes in the cloud environments. In this paper, using tasks execution and resource utilization data extracted from the execution traces of real world applications at Google, we explore the possibility of predicting the scheduling outcome of a task using statistical models. If we can successfully predict tasks failures, we may be able to reduce the execution time of jobs by rescheduling failed tasks earlier (i.e., before their actual failing time). Our results show that statistical models can predict task failures with a precision up to 97.4%, and a recall up to 96.2%. We simulate the potential benefits of such predictions using the tool kit GloudSim and found that they can improve the number of finished tasks by up to 40%. We also perform a case study using the Hadoop framework of Amazon Elastic MapReduce (EMR) and the jobs of a gene expression correlations analysis study from breast cancer research. We find that when extending the scheduler of Hadoop with our predictive models, the percentage of failed jobs can be reduced by up to 45%, with an overhead of less than 5 minutes

Taxonomic classification of planning decisions in health care: a review of the state of the art in OR/MS

We provide a structured overview of the typical decisions to be made in resource capacity planning and control in health care, and a review of relevant OR/MS articles for each planning decision. The contribution of this paper is twofold. First, to position the planning decisions, a taxonomy is presented. This taxonomy provides health care managers and OR/MS researchers with a method to identify, break down and classify planning and control decisions. Second, following the taxonomy, for six health care services, we provide an exhaustive specification of planning and control decisions in resource capacity planning and control. For each planning and control decision, we structurally review the key OR/MS articles and the OR/MS methods and techniques that are applied in the literature to support decision making

Opportunity costs calculation in agent-based vehicle routing and scheduling

In this paper we consider a real-time, dynamic pickup and delivery problem with timewindows where orders should be assigned to one of a set of competing transportation companies. Our approach decomposes the problem into a multi-agent structure where vehicle agents are responsible for the routing and scheduling decisions and the assignment of orders to vehicles is done by using a second-price auction. Therefore the system performance will be heavily dependent on the pricing strategy of the vehicle agents. We propose a pricing strategy for vehicle agents based on dynamic programming where not only the direct cost of a job insertion is taken into account, but also its impact on future opportunities. We also propose a waiting strategy based on the same opportunity valuation. Simulation is used to evaluate the benefit of pricing opportunities compared to simple pricing strategies in different market settings. Numerical results show that the proposed approach provides high quality solutions, in terms of profits, capacity utilization and delivery reliability

Capacity Planning and Leadtime management

In this paper we discuss a framework for capacity planning and lead time management in manufacturing companies, with an emphasis on the machine shop. First we show how queueing models can be used to find approximations of the mean and the variance of manufacturing shop lead times. These quantities often serve as a basis to set a fixed planned lead time in an MRP-controlled environment. A major drawback of a fixed planned lead time is the ignorance of the correlation between actual work loads and the lead times that can be realized under a limited capacity flexibility. To overcome this problem, we develop a method that determines the earliest possible completion time of any arriving job, without sacrificing the delivery performance of any other job in the shop. This earliest completion time is then taken to be the delivery date and thereby determines a workload-dependent planned lead time. We compare this capacity planning procedure with a fixed planned lead time approach (as in MRP), with a procedure in which lead times are estimated based on the amount of work in the shop, and with a workload-oriented release procedure. Numerical experiments so far show an excellent performance of the capacity planning procedure

Heuristics Techniques for Scheduling Problems with Reducing Waiting Time Variance

In real computational world, scheduling is a decision making process. This is nothing but a systematic schedule through which a large numbers of tasks are assigned to the processors. Due to the resource limitation, creation of such schedule is a real challenge. This creates the interest of developing a qualitative scheduler for the processors. These processors are either single or parallel. One of the criteria for improving the efficiency of scheduler is waiting time variance (WTV). Minimizing the WTV of a task is a NP-hard problem. Achieving the quality of service (QoS) in a single or parallel processor by minimizing the WTV is a problem of task scheduling. To enhance the performance of a single or parallel processor, it is required to develop a stable and none overlap scheduler by minimizing WTV. An automated scheduler\u27s performance is always measured by the attributes of QoS. One of the attributes of QoS is ‘Timeliness’. First, this chapter presents the importance of heuristics with five heuristic-based solutions. Then applies these heuristics on 1‖WTV minimization problem and three heuristics with a unique task distribution mechanism on Qm|prec|WTV minimization problem. The experimental result shows the performance of heuristic in the form of graph for consonant problems

Built to Last or Built Too Fast? Evaluating Prediction Models for Build Times

Automated builds are integral to the Continuous Integration (CI) software development practice. In CI, developers are encouraged to integrate early and often. However, long build times can be an issue when integrations are frequent. This research focuses on finding a balance between integrating often and keeping developers productive. We propose and analyze models that can predict the build time of a job. Such models can help developers to better manage their time and tasks. Also, project managers can explore different factors to determine the best setup for a build job that will keep the build wait time to an acceptable level. Software organizations transitioning to CI practices can use the predictive models to anticipate build times before CI is implemented. The research community can modify our predictive models to further understand the factors and relationships affecting build times.Comment: 4 paged version published in the Proceedings of the IEEE/ACM 14th International Conference on Mining Software Repositories (MSR) Pages 487-490. MSR 201