Latency Optimal Broadcasting in Noisy Wireless Mesh Networks

In this paper, we adopt a new noisy wireless network model introduced very recently by Censor-Hillel et al. in [ACM PODC 2017, CHHZ17]. More specifically, for a given noise parameter $p\in [0,1],$ any sender has a probability of $p$ of transmitting noise or any receiver of a single transmission in its neighborhood has a probability $p$ of receiving noise. In this paper, we first propose a new asymptotically latency-optimal approximation algorithm (under faultless model) that can complete single-message broadcasting task in $D+O(\log^2 n)$ time units/rounds in any WMN of size $n,$ and diameter $D$. We then show this diameter-linear broadcasting algorithm remains robust under the noisy wireless network model and also improves the currently best known result in CHHZ17 by a $\Theta(\log\log n)$ factor. In this paper, we also further extend our robust single-message broadcasting algorithm to $k$ multi-message broadcasting scenario and show it can broadcast $k$ messages in $O(D+k\log n+\log^2 n)$ time rounds. This new robust multi-message broadcasting scheme is not only asymptotically optimal but also answers affirmatively the problem left open in CHHZ17 on the existence of an algorithm that is robust to sender and receiver faults and can broadcast $k$ messages in $O(D+k\log n + polylog(n))$ time rounds.Comment: arXiv admin note: text overlap with arXiv:1705.07369 by other author

Learning Scheduling Algorithms for Data Processing Clusters

Efficiently scheduling data processing jobs on distributed compute clusters requires complex algorithms. Current systems, however, use simple generalized heuristics and ignore workload characteristics, since developing and tuning a scheduling policy for each workload is infeasible. In this paper, we show that modern machine learning techniques can generate highly-efficient policies automatically. Decima uses reinforcement learning (RL) and neural networks to learn workload-specific scheduling algorithms without any human instruction beyond a high-level objective such as minimizing average job completion time. Off-the-shelf RL techniques, however, cannot handle the complexity and scale of the scheduling problem. To build Decima, we had to develop new representations for jobs' dependency graphs, design scalable RL models, and invent RL training methods for dealing with continuous stochastic job arrivals. Our prototype integration with Spark on a 25-node cluster shows that Decima improves the average job completion time over hand-tuned scheduling heuristics by at least 21%, achieving up to 2x improvement during periods of high cluster load

A resource allocation mechanism based on cost function synthesis in complex systems

While the management of resources in computer systems can greatly impact the usefulness and integrity of the system, finding an optimal solution to the management problem is unfortunately NP hard. Adding to the complexity, today\u27s \u27modern\u27 systems - such as in multimedia, medical, and military systems - may be, and often are, comprised of interacting real and non-real-time components. In addition, these systems can be driven by a host of non-functional objectives – often differing not only in nature, importance, and form, but also in dimensional units and range, and themselves interacting in complex ways. We refer to systems exhibiting such characteristics as Complex Systems (CS). We present a method for handling the multiple non-functional system objectives in CS, by addressing decomposition, quantification, and evaluation issues. Our method will result in better allocations, improve objective satisfaction, improve the overall performance of the system, and reduce cost -in a global sense. Moreover, we consider the problem of formulating the cost of an allocation driven by system objectives. We start by discussing issues and relationships among global objectives, their decomposition, and cost functions for evaluation of system objective. Then, as an example of objective and cost function development, we introduce the concept of deadline balancing. Next, we proceed by proving the existence of combining models and their underlying conditions. Then, we describe a hierarchical model for system objective function synthesis. This synthesis is performed solely for the purpose of measuring the level of objective satisfaction in a proposed hardware to software allocation, not for design of individual software modules. Then, Examples are given to show how the model applies to actual multi-objective problems. In addition the concept of deadline balancing is extended to a new scheduling concept, namely Inter-Completion-Time Scheduling (ICTS. Finally, experiments based on simulation have been conducted to capture various properties of the synthesis approach as well as ICTS. A prototype implementation of the cost functions synthesis and evaluation environment is described, highlighting the applicability and usefulness of the synthesis in realistic applications

Synthesis of Fault-Tolerant Embedded Systems with Checkpointing and Replication

We present an approach to the synthesis of fault-tolerant hard real-time systems for safety-critical applications. We use checkpointing with rollback recovery and active replication for tolerating transient faults. Processes are statically scheduled and communications are performed using the time-triggered protocol. Our synthesis approach decides the assignment of fault-tolerance policies to processes, the optimal placement of checkpoints and the mapping of processes to processors such that transient faults are tolerated and the timing constraints of the application are satisfied. We present several synthesis algorithms which are able to find fault-tolerant implementations given a limited amount of resources. The developed algorithms are evaluated using extensive experiments, including a real-life example. 1

Design Optimization of Time- and Cost-Constrained Fault-Tolerant Distributed Embedded Systems

Submitted on behalf of EDAA (http://www.edaa.com/)International audienceIn this paper we present an approach to the design optimization of fault-tolerant embedded systems for safety-critical applications. Processes are statically scheduled and communications are performed using the time-triggered protocol. We use process re-execution and replication for tolerating transient faults. Our design optimization approach decides the mapping of processes to processors and the assignment of fault-tolerant policies to processes such that transient faults are tolerated and the timing constraints of the application are satisfied. We present several heuristics which are able to find fault-tolerant implementations given a limited amount of resources. The developed algorithms are evaluated using extensive experiments, including a real-life example