110,548 research outputs found
Revisiting Size-Based Scheduling with Estimated Job Sizes
We study size-based schedulers, and focus on the impact of inaccurate job
size information on response time and fairness. Our intent is to revisit
previous results, which allude to performance degradation for even small errors
on job size estimates, thus limiting the applicability of size-based
schedulers.
We show that scheduling performance is tightly connected to workload
characteristics: in the absence of large skew in the job size distribution,
even extremely imprecise estimates suffice to outperform size-oblivious
disciplines. Instead, when job sizes are heavily skewed, known size-based
disciplines suffer.
In this context, we show -- for the first time -- the dichotomy of
over-estimation versus under-estimation. The former is, in general, less
problematic than the latter, as its effects are localized to individual jobs.
Instead, under-estimation leads to severe problems that may affect a large
number of jobs.
We present an approach to mitigate these problems: our technique requires no
complex modifications to original scheduling policies and performs very well.
To support our claim, we proceed with a simulation-based evaluation that covers
an unprecedented large parameter space, which takes into account a variety of
synthetic and real workloads.
As a consequence, we show that size-based scheduling is practical and
outperforms alternatives in a wide array of use-cases, even in presence of
inaccurate size information.Comment: To be published in the proceedings of IEEE MASCOTS 201
Metascheduling of HPC Jobs in Day-Ahead Electricity Markets
High performance grid computing is a key enabler of large scale collaborative
computational science. With the promise of exascale computing, high performance
grid systems are expected to incur electricity bills that grow super-linearly
over time. In order to achieve cost effectiveness in these systems, it is
essential for the scheduling algorithms to exploit electricity price
variations, both in space and time, that are prevalent in the dynamic
electricity price markets. In this paper, we present a metascheduling algorithm
to optimize the placement of jobs in a compute grid which consumes electricity
from the day-ahead wholesale market. We formulate the scheduling problem as a
Minimum Cost Maximum Flow problem and leverage queue waiting time and
electricity price predictions to accurately estimate the cost of job execution
at a system. Using trace based simulation with real and synthetic workload
traces, and real electricity price data sets, we demonstrate our approach on
two currently operational grids, XSEDE and NorduGrid. Our experimental setup
collectively constitute more than 433K processors spread across 58 compute
systems in 17 geographically distributed locations. Experiments show that our
approach simultaneously optimizes the total electricity cost and the average
response time of the grid, without being unfair to users of the local batch
systems.Comment: Appears in IEEE Transactions on Parallel and Distributed System
Bulk Scheduling with the DIANA Scheduler
Results from the research and development of a Data Intensive and Network
Aware (DIANA) scheduling engine, to be used primarily for data intensive
sciences such as physics analysis, are described. In Grid analyses, tasks can
involve thousands of computing, data handling, and network resources. The
central problem in the scheduling of these resources is the coordinated
management of computation and data at multiple locations and not just data
replication or movement. However, this can prove to be a rather costly
operation and efficient sing can be a challenge if compute and data resources
are mapped without considering network costs. We have implemented an adaptive
algorithm within the so-called DIANA Scheduler which takes into account data
location and size, network performance and computation capability in order to
enable efficient global scheduling. DIANA is a performance-aware and
economy-guided Meta Scheduler. It iteratively allocates each job to the site
that is most likely to produce the best performance as well as optimizing the
global queue for any remaining jobs. Therefore it is equally suitable whether a
single job is being submitted or bulk scheduling is being performed. Results
indicate that considerable performance improvements can be gained by adopting
the DIANA scheduling approach.Comment: 12 pages, 11 figures. To be published in the IEEE Transactions in
Nuclear Science, IEEE Press. 200
Scheduling of data-intensive workloads in a brokered virtualized environment
Providing performance predictability guarantees is increasingly important in cloud platforms, especially for data-intensive applications, for which performance depends greatly on the available rates of data transfer between the various computing/storage hosts underlying the virtualized resources assigned to the application. With the increased prevalence of brokerage services in cloud platforms, there is a need for resource management solutions that consider the brokered nature of these workloads, as well as the special demands of their intra-dependent components. In this paper, we present an offline mechanism for scheduling batches of brokered data-intensive workloads, which can be extended to an online setting. The objective of the mechanism is to decide on a packing of the workloads in a batch that minimizes the broker's incurred costs, Moreover, considering the brokered nature of such workloads, we define a payment model that provides incentives to these workloads to be scheduled as part of a batch, which we analyze theoretically. Finally, we evaluate the proposed scheduling algorithm, and exemplify the fairness of the payment model in practical settings via trace-based experiments
PSBS: Practical Size-Based Scheduling
Size-based schedulers have very desirable performance properties: optimal or
near-optimal response time can be coupled with strong fairness guarantees.
Despite this, such systems are very rarely implemented in practical settings,
because they require knowing a priori the amount of work needed to complete
jobs: this assumption is very difficult to satisfy in concrete systems. It is
definitely more likely to inform the system with an estimate of the job sizes,
but existing studies point to somewhat pessimistic results if existing
scheduler policies are used based on imprecise job size estimations. We take
the goal of designing scheduling policies that are explicitly designed to deal
with inexact job sizes: first, we show that existing size-based schedulers can
have bad performance with inexact job size information when job sizes are
heavily skewed; we show that this issue, and the pessimistic results shown in
the literature, are due to problematic behavior when large jobs are
underestimated. Once the problem is identified, it is possible to amend
existing size-based schedulers to solve the issue. We generalize FSP -- a fair
and efficient size-based scheduling policy -- in order to solve the problem
highlighted above; in addition, our solution deals with different job weights
(that can be assigned to a job independently from its size). We provide an
efficient implementation of the resulting protocol, which we call Practical
Size-Based Scheduler (PSBS). Through simulations evaluated on synthetic and
real workloads, we show that PSBS has near-optimal performance in a large
variety of cases with inaccurate size information, that it performs fairly and
it handles correctly job weights. We believe that this work shows that PSBS is
indeed pratical, and we maintain that it could inspire the design of schedulers
in a wide array of real-world use cases.Comment: arXiv admin note: substantial text overlap with arXiv:1403.599
Efficient heuristics for the parallel blocking flow shop scheduling problem
We consider the NP-hard problem of scheduling n jobs in F identical parallel flow shops, each consisting of a series of m machines, and doing so with a blocking constraint. The applied criterion is to minimize the makespan, i.e., the maximum completion time of all the jobs in F flow shops (lines). The Parallel Flow Shop Scheduling Problem (PFSP) is conceptually similar to another problem known in the literature as the Distributed Permutation Flow Shop Scheduling Problem (DPFSP), which allows modeling the scheduling process in companies with more than one factory, each factory with a flow shop configuration. Therefore, the proposed methods can solve the scheduling problem under the blocking constraint in both situations, which, to the best of our knowledge, has not been studied previously. In this paper, we propose a mathematical model along with some constructive and improvement heuristics to solve the parallel blocking flow shop problem (PBFSP) and thus minimize the maximum completion time among lines. The proposed constructive procedures use two approaches that are totally different from those proposed in the literature. These methods are used as initial solution procedures of an iterated local search (ILS) and an iterated greedy algorithm (IGA), both of which are combined with a variable neighborhood search (VNS). The proposed constructive procedure and the improved methods take into account the characteristics of the problem. The computational evaluation demonstrates that both of them –especially the IGA– perform considerably better than those algorithms adapted from the DPFSP literature.Peer ReviewedPostprint (author's final draft
Scheduling of non-repetitive lean manufacturing systems under uncertainty using intelligent agent simulation
World-class manufacturing paradigms emerge from specific types of manufacturing systems with which they remain associated until they are obsolete. Since its introduction the lean paradigm is almost exclusively implemented in repetitive manufacturing systems employing flow-shop layout configurations. Due to its inherent complexity and combinatorial nature, scheduling is one application domain whereby the implementation of manufacturing philosophies and best practices is particularly challenging. The study of the limited reported attempts to extend leanness into the scheduling of non-repetitive manufacturing systems with functional shop-floor configurations confirms that these works have adopted a similar approach which aims to transform the system mainly through reconfiguration in order to increase the degree of manufacturing repetitiveness and thus facilitate the adoption of leanness. This research proposes the use of leading edge intelligent agent simulation to extend the lean principles and techniques to the scheduling of non-repetitive production environments with functional layouts and no prior reconfiguration of any form. The simulated system is a dynamic job-shop with stochastic order arrivals and processing times operating under a variety of dispatching rules. The modelled job-shop is subject to uncertainty expressed in the form of high priority orders unexpectedly arriving at the system, order cancellations and machine breakdowns. The effect of the various forms of the stochastic disruptions considered in this study on system performance prior and post the introduction of leanness is analysed in terms of a number of time, due date and work-in-progress related performance metrics
The effect of real workloads and stochastic workloads on the performance of allocation and scheduling algorithms in 2D mesh multicomputers
The performance of the existing non-contiguous processor allocation strategies has been traditionally carried out by means of simulation based on a stochastic workload model to generate a stream of incoming jobs. To validate the performance of the existing algorithms, there has been a need to evaluate the algorithms' performance based on a real workload trace. In this paper, we evaluate the performance of several well-known processor allocation and job scheduling strategies based on a real workload trace and compare the results against those obtained from using a stochastic workload. Our results reveal that the conclusions reached on the relative performance merits of the allocation strategies when a real workload trace is used are in general compatible with those obtained when a stochastic workload is used
- …