How Workflow Engines Should Talk to Resource Managers: A Proposal for a Common Workflow Scheduling Interface

Scientific workflow management systems (SWMSs) and resource managers together ensure that tasks are scheduled on provisioned resources so that all dependencies are obeyed, and some optimization goal, such as makespan minimization, is fulfilled. In practice, however, there is no clear separation of scheduling responsibilities between an SWMS and a resource manager because there exists no agreed-upon separation of concerns between their different components. This has two consequences. First, the lack of a standardized API to exchange scheduling information between SWMSs and resource managers hinders portability. It incurs costly adaptations when a component should be replaced by another one (e.g., an SWMS with another SWMS on the same resource manager). Second, due to overlapping functionalities, current installations often actually have two schedulers, both making partial scheduling decisions under incomplete information, leading to suboptimal workflow scheduling. In this paper, we propose a simple REST interface between SWMSs and resource managers, which allows any SWMS to pass dynamic workflow information to a resource manager, enabling maximally informed scheduling decisions. We provide an exemplary implementation of this API for Nextflow as an SWMS and Kubernetes as a resource manager. Our experiments with nine real-world workflows show that this strategy reduces makespan by up to 25.1% and 10.8% on average compared to the standard Nextflow/Kubernetes configuration. Furthermore, a more widespread implementation of this API would enable leaner code bases, a simpler exchange of components of workflow systems, and a unified place to implement new scheduling algorithms.Comment: Paper accepted in: 2023 23rd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid

A survey on gain-scheduled control and filtering for parameter-varying systems

Copyright © 2014 Guoliang Wei et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.This paper presents an overview of the recent developments in the gain-scheduled control and filtering problems for the parameter-varying systems. First of all, we recall several important algorithms suitable for gain-scheduling method including gain-scheduled proportional-integral derivative (PID) control, H 2, H ∞ and mixed H 2 / H ∞ gain-scheduling methods as well as fuzzy gain-scheduling techniques. Secondly, various important parameter-varying system models are reviewed, for which gain-scheduled control and filtering issues are usually dealt with. In particular, in view of the randomly occurring phenomena with time-varying probability distributions, some results of our recent work based on the probability-dependent gain-scheduling methods are reviewed. Furthermore, some latest progress in this area is discussed. Finally, conclusions are drawn and several potential future research directions are outlined.The National Natural Science Foundation of China under Grants 61074016, 61374039, 61304010, and 61329301; the Natural Science Foundation of Jiangsu Province of China under Grant BK20130766; the Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning; the Program for New Century Excellent Talents in University under Grant NCET-11-1051, the Leverhulme Trust of the U.K., the Alexander von Humboldt Foundation of Germany

Scheduling with Rate Adaptation under Incomplete Knowledge of Channel/Estimator Statistics

In time-varying wireless networks, the states of the communication channels are subject to random variations, and hence need to be estimated for efficient rate adaptation and scheduling. The estimation mechanism possesses inaccuracies that need to be tackled in a probabilistic framework. In this work, we study scheduling with rate adaptation in single-hop queueing networks under two levels of channel uncertainty: when the channel estimates are inaccurate but complete knowledge of the channel/estimator joint statistics is available at the scheduler; and when the knowledge of the joint statistics is incomplete. In the former case, we characterize the network stability region and show that a maximum-weight type scheduling policy is throughput-optimal. In the latter case, we propose a joint channel statistics learning - scheduling policy. With an associated trade-off in average packet delay and convergence time, the proposed policy has a stability region arbitrarily close to the stability region of the network under full knowledge of channel/estimator joint statistics.Comment: 48th Allerton Conference on Communication, Control, and Computing, Monticello, IL, Sept. 201

A Structural Safety Analysis of Buildings During Construction

The safety of steel buildings, constructed by the tier method, is evaluated. The probability of failure of steel frames supported on temporary connections is examined during the different stages of completion. The principal loading of concern is the maximum wind load over the critical stages of construction.National Science Foundation Grants ENG 77-02007, ENV 77-09090, and PFR 80-0258

When queueing is better than push and shove

We address the scheduling problem of reordering an existing queue into its efficient order through trade. To that end, we consider individually rational and balanced budget direct and indirect mechanisms. We show that this class of mechanisms allows us to form efficient queues provided that existing property rights for the service are small enough to enable trade between the agents. In particular, we show on the one hand that no queue under a fully deterministic service schedule such as first-come, first-serve can be dissolved efficiently and meet our requirements. If, on the other hand, the alternative is service anarchy (ie. a random queue), every existing queue can be transformed into an efficient order