Scheduling of coupled tasks with exact delays for minimum total job completion time

In this paper, we fill in a conspicuous gap in research on scheduling coupled tasks. We draw a full complexity picture for single-machine scheduling of coupled tasks of exact time delays in between with the objective of minimizing the total of job completion times

The Coupled Task Scheduling Problem: Models and Solution Methods

University of Technology Sydney. Faculty of Science.The coupled task scheduling problem (CTSP) is studied in this thesis. The problem consists of scheduling a set of jobs on one or a set of machines, where each job consists of at least two tasks. The main characteristic of the problem is a fixed time-lag between the process of each two consecutive tasks of the same job, where its duration is fixed, i.e., the succeeding task cannot be started earlier or later than the time-lag is passed. The fixed time-lags were introduced to model radar tracking systems, and later extended to formulate problems in chemistry manufacturing systems and robotic cells. The motivation for studying the CTSP in this thesis is to model certain problems in healthcare scheduling with the same characteristics. One example is the scheduling of patients in a chemotherapy clinic, where each patient must undergo a number of consecutive treatments with time-lags in between. Meeting the fixed delays between the treatments of a patient is an important factor in gaining the best outcomes for them. To study the CTSP, a literature review is first conducted, followed by studying the problem in different scheduling environments, including the single-machine, parallel-machine, open-shop and flow-shop settings, where we propose several new complexity results and solution algorithms for different variants of the problem. Regarding the single-machine coupled task problem, a new mathematical formulation and two matheuristic algorithms are proposed for the classical problem, as well as a dynamic programming algorithm for a variant of the problem with time-dependent processing times. With regard to the parallel-machine environment, we first explore the complexity of the problem and propose NP-hardness proofs for certain cases, followed by approximation bounds for the two-machine problem. The latter result is then extended to the open-shop scheduling environment. The problem in the flow-shop environment is then extensively investigated under the permutation setting, and also under the case of ordered processing times. A set of publicly available hard data set and state-of-the-art algorithms are proposed for the ordered flow-shops. Then, flow-shop problem with coupled tasks is studied and polynomial-time algorithms are proposed for various settings of the problem, including the ordered processing times

A Tale of Two Data-Intensive Paradigms: Applications, Abstractions, and Architectures

Scientific problems that depend on processing large amounts of data require overcoming challenges in multiple areas: managing large-scale data distribution, co-placement and scheduling of data with compute resources, and storing and transferring large volumes of data. We analyze the ecosystems of the two prominent paradigms for data-intensive applications, hereafter referred to as the high-performance computing and the Apache-Hadoop paradigm. We propose a basis, common terminology and functional factors upon which to analyze the two approaches of both paradigms. We discuss the concept of "Big Data Ogres" and their facets as means of understanding and characterizing the most common application workloads found across the two paradigms. We then discuss the salient features of the two paradigms, and compare and contrast the two approaches. Specifically, we examine common implementation/approaches of these paradigms, shed light upon the reasons for their current "architecture" and discuss some typical workloads that utilize them. In spite of the significant software distinctions, we believe there is architectural similarity. We discuss the potential integration of different implementations, across the different levels and components. Our comparison progresses from a fully qualitative examination of the two paradigms, to a semi-quantitative methodology. We use a simple and broadly used Ogre (K-means clustering), characterize its performance on a range of representative platforms, covering several implementations from both paradigms. Our experiments provide an insight into the relative strengths of the two paradigms. We propose that the set of Ogres will serve as a benchmark to evaluate the two paradigms along different dimensions.Comment: 8 pages, 2 figure

Towards Loosely-Coupled Programming on Petascale Systems

We have extended the Falkon lightweight task execution framework to make loosely coupled programming on petascale systems a practical and useful programming model. This work studies and measures the performance factors involved in applying this approach to enable the use of petascale systems by a broader user community, and with greater ease. Our work enables the execution of highly parallel computations composed of loosely coupled serial jobs with no modifications to the respective applications. This approach allows a new-and potentially far larger-class of applications to leverage petascale systems, such as the IBM Blue Gene/P supercomputer. We present the challenges of I/O performance encountered in making this model practical, and show results using both microbenchmarks and real applications from two domains: economic energy modeling and molecular dynamics. Our benchmarks show that we can scale up to 160K processor-cores with high efficiency, and can achieve sustained execution rates of thousands of tasks per second.Comment: IEEE/ACM International Conference for High Performance Computing, Networking, Storage and Analysis (SuperComputing/SC) 200

Many-Task Computing and Blue Waters

This report discusses many-task computing (MTC) generically and in the context of the proposed Blue Waters systems, which is planned to be the largest NSF-funded supercomputer when it begins production use in 2012. The aim of this report is to inform the BW project about MTC, including understanding aspects of MTC applications that can be used to characterize the domain and understanding the implications of these aspects to middleware and policies. Many MTC applications do not neatly fit the stereotypes of high-performance computing (HPC) or high-throughput computing (HTC) applications. Like HTC applications, by definition MTC applications are structured as graphs of discrete tasks, with explicit input and output dependencies forming the graph edges. However, MTC applications have significant features that distinguish them from typical HTC applications. In particular, different engineering constraints for hardware and software must be met in order to support these applications. HTC applications have traditionally run on platforms such as grids and clusters, through either workflow systems or parallel programming systems. MTC applications, in contrast, will often demand a short time to solution, may be communication intensive or data intensive, and may comprise very short tasks. Therefore, hardware and software for MTC must be engineered to support the additional communication and I/O and must minimize task dispatch overheads. The hardware of large-scale HPC systems, with its high degree of parallelism and support for intensive communication, is well suited for MTC applications. However, HPC systems often lack a dynamic resource-provisioning feature, are not ideal for task communication via the file system, and have an I/O system that is not optimized for MTC-style applications. Hence, additional software support is likely to be required to gain full benefit from the HPC hardware

Squeaky Wheel Optimization

We describe a general approach to optimization which we term `Squeaky Wheel' Optimization (SWO). In SWO, a greedy algorithm is used to construct a solution which is then analyzed to find the trouble spots, i.e., those elements, that, if improved, are likely to improve the objective function score. The results of the analysis are used to generate new priorities that determine the order in which the greedy algorithm constructs the next solution. This Construct/Analyze/Prioritize cycle continues until some limit is reached, or an acceptable solution is found. SWO can be viewed as operating on two search spaces: solutions and prioritizations. Successive solutions are only indirectly related, via the re-prioritization that results from analyzing the prior solution. Similarly, successive prioritizations are generated by constructing and analyzing solutions. This `coupled search' has some interesting properties, which we discuss. We report encouraging experimental results on two domains, scheduling problems that arise in fiber-optic cable manufacturing, and graph coloring problems. The fact that these domains are very different supports our claim that SWO is a general technique for optimization

Architecture independent environment for developing engineering software on MIMD computers

Engineers are constantly faced with solving problems of increasing complexity and detail. Multiple Instruction stream Multiple Data stream (MIMD) computers have been developed to overcome the performance limitations of serial computers. The hardware architectures of MIMD computers vary considerably and are much more sophisticated than serial computers. Developing large scale software for a variety of MIMD computers is difficult and expensive. There is a need to provide tools that facilitate programming these machines. First, the issues that must be considered to develop those tools are examined. The two main areas of concern were architecture independence and data management. Architecture independent software facilitates software portability and improves the longevity and utility of the software product. It provides some form of insurance for the investment of time and effort that goes into developing the software. The management of data is a crucial aspect of solving large engineering problems. It must be considered in light of the new hardware organizations that are available. Second, the functional design and implementation of a software environment that facilitates developing architecture independent software for large engineering applications are described. The topics of discussion include: a description of the model that supports the development of architecture independent software; identifying and exploiting concurrency within the application program; data coherence; engineering data base and memory management

Practical solutions for a dock assignment problem with trailer transportation.

We study a distribution warehouse in which trailers need to be assigned to docks for loading or unloading. A parking lot is used as a buffer zone and transportation between the parking lot and the docks is performed by auxiliary resources called terminal tractors. Each incoming trailer has a known arrival time and each outgoing trailer a desired departure time. The primary objective is to produce a docking schedule such that the weighted sum of the number of late outgoing trailers and the tardiness of these trailers is minimized; the secondary objective is to minimize the weighted completion time of all trailers, both incoming and outgoing. The purpose of this paper is to produce high-quality solutions to large instances that are comparable to a real-life case. We implement several heuristic algorithms: truncated branch and bound, beam search and tabu search. Lagrangian relaxation is embedded in the algorithms for constructing an initial solution and for computing lower bounds. The different solution frameworks are compared via extensive computational experiments.Dock assignment; Multicriteria scheduling; Branch and bound; Beam search; Lagrangian relaxation; Tabu search;