QoS and security-aware task assignment and scheduling in real-time systems

Security issues in mission-critical real-time systems (e.g., command and control systems) are becoming increasingly important as there are growing needs for satisfying information assurance in these systems. In such systems, it is important to guarantee real-time deadlines along with the security requirements (e.g., confidentiality, integrity, and availability) of the applications. Traditionally, resource management in real-time systems has focused on meeting deadlines along with satisfying fault-tolerance and/or resource constraints. Such an approach is inadequate to accommodate security requirements into resource management algorithms. Based on the imprecise computation paradigm, a task can have several Quality of Service (QoS) levels, higher QoS result incurs higher computational cost. Similarly, achieving a higher level of confidentially requires stronger encryption, which incurs higher computational cost. Therefore, there exists a tradeoff between schedulability of the tasks on the one hand, and the accuracy (QoS) and security of the results produced on the other hand. This tradeoff must be carefully accounted in the resource management algorithms. In this context, this dissertation makes the following contributions: (i) formulation of scheduling problems accounting both deadline and security requirements of workloads in real-time systems, (ii) development of novel task allocation and scheduling algorithms for such workloads, (iii) and evaluation of the results through simulation studies and a limited test evaluations in one case. In particular, the following are the three key contributions. Firstly, the problem of scheduling a set of non-preemptable real-time tasks with security and QoS requirements with the goal of maximizing integrated QoS and security of the system is addressed. This problem is formulated as MILP, and then its complexity is proved to be NP-hard. An online efficient heuristic algorithm is developed as the problem is NP-hard. Simulation studies for a wide range of workload scenarios showed that the proposed algorithm outperforms a set of baseline algorithms. Further, the proposed algorithm\u27s performance is close to the optimal solution in a specific special case of the problem. Secondly, a static assignment and scheduling of a set of dependent real-time tasks, modeled as Directed Acyclic Graph (DAG), with security and QoS requirements in heterogeneous real-time system with the objective of maximizing Total Quality Value (TQV) of the system is studied. This problem is formulated as MINLP. Since this problem is NP-hard, a heuristic algorithm to maximize TQV while satisfying the security constraint of the system is developed. The proposed algorithm was evaluated through extensive simulation studies and compared to a set of baseline algorithms for variations of synthetic workloads. The proposed algorithm outperforms the baseline algorithms in all the simulated conditions for fully-connected and shared bus network topologies. Finally, the problem of dynamic assignment and scheduling of a set of dependent tasks with QoS and security requirements in heterogeneous distributed system to maximize the system TQV is addressed. Two heuristic algorithms to maximize TQV of the system are proposed because the problem is NP-hard. The proposed algorithms were evaluated by extensive simulation studies and by a test experiment in InfoSpher platform. The proposed algorithms outperform the baseline algorithms in most of the simulated conditions for fully-connected and shared bus network topologies

Fair Resource Sharing for Dynamic Scheduling of Workflows on Heterogeneous Systems

International audienceScheduling independent workflows on shared resources in a way that satisfy users Quality of Service is a significant challenge. In this study, we describe methodologies for off-line scheduling, where a schedule is generated for a set of knownworkflows, and on-line scheduling, where users can submit workflows at any moment in time. We consider the on-line scheduling problem in more detail and present performance comparisons of state-of-the-art algorithms for a realistic model of a heterogeneous system

How the structure of precedence constraints may change the complexity class of scheduling problems

This survey aims at demonstrating that the structure of precedence constraints plays a tremendous role on the complexity of scheduling problems. Indeed many problems can be NP-hard when considering general precedence constraints, while they become polynomially solvable for particular precedence constraints. We also show that there still are many very exciting challenges in this research area

Scientific Computing Meets Big Data Technology: An Astronomy Use Case

Scientific analyses commonly compose multiple single-process programs into a dataflow. An end-to-end dataflow of single-process programs is known as a many-task application. Typically, tools from the HPC software stack are used to parallelize these analyses. In this work, we investigate an alternate approach that uses Apache Spark -- a modern big data platform -- to parallelize many-task applications. We present Kira, a flexible and distributed astronomy image processing toolkit using Apache Spark. We then use the Kira toolkit to implement a Source Extractor application for astronomy images, called Kira SE. With Kira SE as the use case, we study the programming flexibility, dataflow richness, scheduling capacity and performance of Apache Spark running on the EC2 cloud. By exploiting data locality, Kira SE achieves a 2.5x speedup over an equivalent C program when analyzing a 1TB dataset using 512 cores on the Amazon EC2 cloud. Furthermore, we show that by leveraging software originally designed for big data infrastructure, Kira SE achieves competitive performance to the C implementation running on the NERSC Edison supercomputer. Our experience with Kira indicates that emerging Big Data platforms such as Apache Spark are a performant alternative for many-task scientific applications

Approximation Algorithms for Correlated Knapsacks and Non-Martingale Bandits

In the stochastic knapsack problem, we are given a knapsack of size B, and a set of jobs whose sizes and rewards are drawn from a known probability distribution. However, we know the actual size and reward only when the job completes. How should we schedule jobs to maximize the expected total reward? We know O(1)-approximations when we assume that (i) rewards and sizes are independent random variables, and (ii) we cannot prematurely cancel jobs. What can we say when either or both of these assumptions are changed? The stochastic knapsack problem is of interest in its own right, but techniques developed for it are applicable to other stochastic packing problems. Indeed, ideas for this problem have been useful for budgeted learning problems, where one is given several arms which evolve in a specified stochastic fashion with each pull, and the goal is to pull the arms a total of B times to maximize the reward obtained. Much recent work on this problem focus on the case when the evolution of the arms follows a martingale, i.e., when the expected reward from the future is the same as the reward at the current state. What can we say when the rewards do not form a martingale? In this paper, we give constant-factor approximation algorithms for the stochastic knapsack problem with correlations and/or cancellations, and also for budgeted learning problems where the martingale condition is not satisfied. Indeed, we can show that previously proposed LP relaxations have large integrality gaps. We propose new time-indexed LP relaxations, and convert the fractional solutions into distributions over strategies, and then use the LP values and the time ordering information from these strategies to devise a randomized adaptive scheduling algorithm. We hope our LP formulation and decomposition methods may provide a new way to address other correlated bandit problems with more general contexts

A Survey of Pipelined Workflow Scheduling: Models and Algorithms

International audienceA large class of applications need to execute the same workflow on different data sets of identical size. Efficient execution of such applications necessitates intelligent distribution of the application components and tasks on a parallel machine, and the execution can be orchestrated by utilizing task-, data-, pipelined-, and/or replicated-parallelism. The scheduling problem that encompasses all of these techniques is called pipelined workflow scheduling, and it has been widely studied in the last decade. Multiple models and algorithms have flourished to tackle various programming paradigms, constraints, machine behaviors or optimization goals. This paper surveys the field by summing up and structuring known results and approaches