Agile Development of Linux Schedulers with Ekiben

Kernel task scheduling is important for application performance, adaptability to new hardware, and complex user requirements. However, developing, testing, and debugging new scheduling algorithms in Linux, the most widely used cloud operating system, is slow and difficult. We developed Ekiben, a framework for high velocity development of Linux kernel schedulers. Ekiben schedulers are written in safe Rust, and the system supports live upgrade of new scheduling policies into the kernel, userspace debugging, and bidirectional communication with applications. A scheduler implemented with Ekiben achieved near identical performance (within 1% on average) to the default Linux scheduler CFS on a wide range of benchmarks. Ekiben is also able to support a range of research schedulers, specifically the Shinjuku scheduler, a locality aware scheduler, and the Arachne core arbiter, with good performance.Comment: 13 pages, 5 figures, submitted to Eurosys 202

Implementing and Evaluating Jukebox Schedulers Using JukeTools

Scheduling jukebox resources is important to build efficient and flexible hierarchical storage systems. JukeTools is a toolbox that helps in the complex tasks of implementing and evaluating jukebox schedulers. It allows the fast development of jukebox schedulers. The schedulers can be tested in numerous environments, both real and simulated types. JukeTools helps the developer to easily detect errors in the schedules. Analyzer tools create detailed reports on the behavior and performance of any of the scheduler, and provide comparisons between different schedulers. This paper describes the functionality offered by JukeTools, with special emphasis on how the toolbox can be used to develop jukebox schedulers

Extending snBench to Support Hierarchical and Configurable Scheduling

It is useful in systems that must support multiple applications with various temporal requirements to allow application-specific policies to manage resources accordingly. However, there is a tension between this goal and the desire to control and police possibly malicious programs. The Java-based Sensor Execution Environment (SXE) in snBench presents a situation where such considerations add value to the system. Multiple applications can be run by multiple users with varied temporal requirements, some Real-Time and others best effort. This paper outlines and documents an implementation of a hierarchical and configurable scheduling system with which different applications can be executed using application-specific scheduling policies. Concurrently the system administrator can define fairness policies between applications that are imposed upon the system. Additionally, to ensure forward progress of system execution in the face of malicious or malformed user programs, an infrastructure for execution using multiple threads is described

How to rapidly prototype a real-time scheduler

Journal ArticleImplementing a new scheduling algorithm in an OS kernel is often an important step in scheduling research because it permits evaluation of the algorithm's performance on real workloads. However, developing a new scheduler is not a trivial task because it requires sophisticated programming skills and a deep knowledge of kernel internals. In this paper we show how to use the HLS scheduling framework to develop new schedulers in a user-level simulator, where advanced debugging tools can be used to achieve a high level of robustness before the scheduler is converted to a loadable kernel module simply by recompiling it. Besides facilitating debugging and porting, the HLS abstraction has the benefit of bringing the programming model very close to what, in our experience, scheduler developers want

A Practical Framework for Configuration of Scheduling and Concurrency Control in Linux

There is a growing need for developers to be able to specify programming models for an application, in order to: increase efficiency, system reliability, system security, and to allow applications with different semantics to coexist on the same system. Only specifying the scheduling semantics for an application is not sufficient because concurrency control also significantly affects the behavior of the application. This work demonstrates the integration of the Hierarchical Group Scheduling and Proxy Management frameworks to provide the ability to developers to configure scheduling and concurrency control semantics for a wide range of applications. This work targets the Linux platform to be useful to a large audience of developers. Additionally, an environment for verifying the correctness of this integration and other concurrent applications using deterministic testing is discussed

Efficient Resource Matching in Heterogeneous Grid Using Resource Vector

In this paper, a method for efficient scheduling to obtain optimum job throughput in a distributed campus grid environment is presented; Traditional job schedulers determine job scheduling using user and job resource attributes. User attributes are related to current usage, historical usage, user priority and project access. Job resource attributes mainly comprise of soft requirements (compilers, libraries) and hard requirements like memory, storage and interconnect. A job scheduler dispatches jobs to a resource if a job's hard and soft requirements are met by a resource. In current scenario during execution of a job, if a resource becomes unavailable, schedulers are presented with limited options, namely re-queuing job or migrating job to a different resource. Both options are expensive in terms of data and compute time. These situations can be avoided, if the often ignored factor, availability time of a resource in a grid environment is considered. We propose resource rank approach, in which jobs are dispatched to a resource which has the highest rank among all resources that match the job's requirement. The results show that our approach can increase throughput of many serial / monolithic jobs.Comment: 10 page